Conclusion Report “An Outsider’s Summary” by Reto Krummenacher: some conclusions from the Platform Workshop about the role of the Executor Component (aka Flow Controller) and other related components which are consequence of a more elaborated discussion
Some thoughts about the document mentioned above and topics for discussion
page 4: "A very simple process description formalism (...) developed in the SOA4All project"
- Is this already available? How big will be the effort/risk to integrate a non-LarKC piece (which is under development) within the LarKC platform?
- A simple but not final RDF schema is available. There are always risks with research projects - and considering LarKC to be a development project in this respect, we should not to take too many risks and to do too much research in context of the implementation. This only makes sense if we reconsider the entire implementation, what is not advisable, as there is a lot of very complex code in the platform already.
- How much is this limiting us to use, in future, “state-of-the-art" workflow enactors (this was one of the comments in our first review, whether we could plug any Decider implementing a different workflow enactor)?
- This RDF schema is built around standard concepts of processes, activities, services, choices, inputs and outputs... however, it is not linked to any standard workflow engine. It is entirely defined within a sub-project of soa4all.
- However, the current platform implementation would not allow plugging any workflow enactor as decider right now, since the decider does not have any standard input or output types either.
- When we planned the architecture, we thought it can be a good idea to keep potential compatibility with existing workflow engines. However, if we now decide that this would add too much complexity, and we implement a different solution which may add other advantages to the platform, it would also be ok.
- Defining a clear workflow description document (using the SOA4All RDF schema, or another one), which should be the output of the Decider towards the Executor, would be the optimal way to implement it. This would also ease the life of Decider plug-in developers, which would have a clear specification to follow.
- The proposal of clearly separating workflow planning from workflow execution would be a first step in that direction. Currently the decider functionality is mixed up with the workflow enactor functionality. If the decider would provide as output a standard workflow description, it would be really easy to address this concern by the reviewer:
decider -> workflow -> executor
- if the workflow is standard, then any executor could do what follows the standard. However currently we have all in one very big and complex box, which will be hard to open in the first place. Christoph is already working on that - but it will be tough.
- In this case a standard workflow enactor would be used as part of the executor. Is SOA4All using any concrete one?
- Yes, and yes - however, the RDFS mentioned above is not compatible with the executor work in soa4all. And we still don’t have details, as it is really hard to get to the bottom of the work that is done around execution in soa4all...
- Is this already available? How big will be the effort/risk to integrate a non-LarKC piece (which is under development) within the LarKC platform?
Page 5: "Management of the execution environment": does this include selection of execution environment resources where each plug-in can/should be executed? And triggering the corresponding remote execution (by e.g. invoking remote plug-in manager)?
- This is related to the "plumbing of plug-ins": this would cover the aspects abovementioned. Still, the decision of whether to invoke a remote plug-in or a local one is subject to the decider - it is a design decision. The executor only takes care of enacting a workflow.
Section 1.3 Revised Logical Architecture:
- In one of the first architecture designs we had exactly this architecture, where the Executor was called Pipeline Support System (where pipeline was what is currently known as workflow). It was supposed to be in charge of the instantiation of plug-ins according to the workflow description provided to him by the Decider, and notify him about the results/alerts of plug-ins execution, to allow the Decider to take intermediate decisions. In the rapid prototyping phase, this component was dropped as such, and we never recovered it again. I think it’s time to do so.
- Taking existing implemented Deciders and extracting from them the features which are not exclusively taking decisions could help to figure out which features should be transferred to the platform components. We end somehow up with two distinct projects:
- separating decider and executor, and
- implementing more complex workflows (split/merge to start)
- The Executer will need different "plumbing components" depending on the kind of split/join. During the Stuttgart workshop we had this discussion. They can be seen as 2 different problems, which should be implemented as separate components which are used by the Executor itself:
- Flow Split/Join: In order to branch workflows, it is necessary to control the flow in the sense of invoking several plug-ins at the same time (parallelization "across plug-ins") and synchronizing their execution time (we may not want to invoke a next plug-in until the execution of the two parallel branches is finished). For this purpose, a mechanism for synchronization of plug-ins execution is needed.
- Data Split/Join: In certain cases, two branches in a workflow may require to process different chunks of the same dataset. For this purpose, a component that supports this splitting of data is needed. When the execution of two parallel branches is finished, the results of them may need to be joint in a unique dataset, which will be input for the next plug-in. For this purpose, a component which supports joining data sets is needed.
- To summarize, the Executor issue can be seen as 3 different projects:
- decider / executor split
- flow split / join
- data split / join
Section 2 Context and Contract: Do you intend to store in the context map also the QoS requirements? The Contract parameter was thought to contain the QoS requirements under which the plug-in or workflow should run. The user should be able to specify his/her own requirements (e.g. I want a first result in less than 1 minute, or I want only a maximum of 10 results), additionally to the query (I guess this would violate the SPARQL protocol, right? Or should the QoS requirements be part of the Sparql query?). The Decider would then translate (break down) the user requirements into QoS constraints on the plug-ins which are part of the workflow. E.g.: a given max. response time dictated by the user is translated to a max. number of triples to retrieve by the Select plug-in, in order to limit the required computation time by the Reason plug-in. (Some of) those values will be passed on to the plug-ins at the time of invocation, when applicable.
- Yes, the contractual parameters which should be derived from the user requirements by the decider would become part of the execution context, and hence also part of the "input state map". This of course implies that the decider is able to translate user requirements into execution context which the plug-ins understand. Quite a job for the decider... and right, this cannot be part of the sparql query.
Section 4 SPARQL endpoint: is there any possibility to extend the SPARQL protocol to be able to get streaming results? This could be pushed to standardisation.
- This is a question for the streaming people... CEFRIEL, Poli Milano, VUA?
Section 5 LarKC and SOA4All
- Possibility to publish LarKC services in SOA4All: during the Stutt meeting you said that a "standard" WSDL would be enough to do that and that any Java plug-in would work, without the need to be a web service. Is this correct? If so, can you please tell us which are the requirements and procedure to have our LarKC plug-ins published in iServe?
- Annotating java-only plug-ins would be doable, as they have a WSDL as well, but it would not make much sense conceptually. Annotating a web service that is not a web service does not help much. Hence, it would likely only make really sense for plug-ins that are wrapped as servlets and that are thus remotely invokable. What is needed: an annotated WSDL according to SA-WSDL and WSMO-Lite.
If the current plug-ins annotations are correct, then we are ready to register our plug-ins in SOA4All. please check the browser at http://iserve-dev.kmi.open.ac.uk:8080/iserve/ and the REST API specification at http://iserve.kmi.open.ac.uk/
- "significant basis of tools and technicalities...": in our first larkc-soa4all meeting, we identified several potential points of collaboration but there was a big risk to use complete components, since the timelines of both projects are not really compatible. In terms of conceptual work, I am really happy to collaborate. In terms of using sw components, I am a bit more reluctant, due to the risk to create dangerous dependencies. E.g. about the tool for constructing processes, Barry checked its availability before leaving, and he decided that the best solution is to use our own workflow editor (a first version is being already shipped with the 1.0 release). If you know any other tool we could use, please let us know.
- Possibility to publish LarKC services in SOA4All: during the Stutt meeting you said that a "standard" WSDL would be enough to do that and that any Java plug-in would work, without the need to be a web service. Is this correct? If so, can you please tell us which are the requirements and procedure to have our LarKC plug-ins published in iServe?