This commit is contained in:
NikolajDanger
2023-05-16 11:17:55 +02:00
parent 238296b618
commit f28865a92e
2 changed files with 19 additions and 3 deletions

Binary file not shown.

View File

@ -42,6 +42,7 @@
\begin{itemize}
\item What work am I doing on MEOW?
\item How did it go?
\item Introduce the concept of network events.
\item \textbf{Write this last}
\end{itemize}
\end{tcolorbox}
@ -54,10 +55,17 @@
Firstly, network event triggers would allow for manual triggering of jobs remotely, without the need for direct access to the monitored files. This is particularly useful in scenarios where human intervention or decision-making is required before proceeding with the subsequent steps in a workflow. While it is possible to manually trigger job using file events by making changes to the monitored directories, this might lead to an already running job accessing the files at the same time, which could cause problems with data integrity.
Secondly, incorporating network event triggers would facilitate seamless communication between parallel jobs, ensuring that tasks can efficiently exchange information and synchronize their progress.
Secondly, incorporating network event triggers would facilitate seamless communication between parallel runners, ensuring that tasks can efficiently exchange information and synchronize their progress.
Finally, extending MEOW's event-based scheduler to support network event triggers would enable the simple and efficient exchange of data between workflows running on different machines. This feature is particularly valuable in distributed computing environments, where data processing tasks are often split across multiple systems to maximize resource utilization and minimize latency. By leveraging network event triggers, MEOW would be better equipped to manage complex workflows in these environments, ensuring seamless integration and streamlined data processing
Finally, extending MEOW's event-based scheduler to support network event triggers would enable the simple and efficient exchange of data between workflows running on different machines. This feature is particularly valuable in distributed computing environments, where data processing tasks are often split across multiple systems to maximize resource utilization and minimize latency.
\begin{tcolorbox}[colback=lightgray!30!white]
How exactly would network events improve things? Expand using graph David sent to me.
\end{tcolorbox}
\begin{tcolorbox}[colback=lightgray!30!white]
Move this to after method (Future work).
\end{tcolorbox}
One specific example of a use-case where network event triggers could prove useful is the workflow for The Brain Imaging Data Structure (BIDS). The BIDS workflow requires data to be sent between multiple machines and validated by a user. Network event triggers could streamline this process by automatically initiating data transfer tasks when specific conditions are met, thereby reducing the need for manual management. Additionally, network triggers could facilitate user validation by allowing users to manually prompt the continuation of the workflow through specific network requests, simplifying the user's role in the validation process.
\begin{figure}[H]
@ -69,6 +77,11 @@
\subsection{Background}
\subsubsection{The structure of MEOW}
\begin{tcolorbox}[colback=lightgray!30!white]
Expand. More diagrams.
\end{tcolorbox}
The MEOW event-based scheduler has three main parts: \textit{monitors}, \textit{handlers}, and \textit{the conductor}.
\begin{figure}[H]
@ -81,9 +94,12 @@
\end{center}
\caption{\textbf{WIP.} How the three elements of MEOW interact.}
\end{figure}
\begin{tcolorbox}[colback=lightgray!30!white]
Include the runner in diagram.
\end{tcolorbox}
Monitors monitor for triggering events. They are initialized with a number of \textit{patterns}, which describe the triggering event. When a pattern's triggering event occurs, the monitor signals to the conductor that the pattern has been triggered.
Monitors listen for triggering events. They are initialized with a number of \textit{patterns}, which describe the triggering event. When a pattern's triggering event occurs, the monitor signals to the conductor that the pattern has been triggered.
Handlers perform actions and jobs on behalf of the scheduler. They are initialized with a number of \textit{recipes}, which describe the action to be taken. The handler starts a job when signal to do so by the conductor.