Working with isolates and namespaces

[mfleming]

I'm attempting to use isolates to speed up an application, but I'm having trouble with references to functions in a namespace. Here's a simple example to illustrate the problem. Suppose I have three functions at the root of my workspace like this

main←{(⍵⍴⍺)foo¨⍳⍵}
foo←{⍺ bar ⍵}
bar←{⍺ × ⍵}

4 main 5 would then return 4 8 12 16 20. Now if I want to speed this up using isolates, I would do something like

)copy isolate
isolate.Config 'listen' 1

main←{(⍵⍴⍺)foo ll.Each ⍳⍵}
foo←{⍺ ##.bar ⍵}
bar←{⍺ × ⍵}

The isolate needs the namespace path for bar, otherwise I get an error. The problem arises when bar is a function in a namespace other than the root. Per the documentation, in an isolate ## refers to the root of the workspace that spawned the isolate. So if my functions are in a namespace called test, I'd need to define foo as

foo←{⍺ ##.test.bar ⍵}

Hardcoding a namespace path for all functions referenced in an isolate seems a bad idea! Is there some way to give an isolate the equivalent of ⎕PATH to resolve a reference to a function name?

Thanks in advance,
~Mark

[Morten|Dyalog]

Mark, apologies for the slow response, I was travelling and having trouble keeping up with incoming messages.

It would probably be possible to allow a reference to ##.bar to use ⎕PATH in the parent space and thus get your example to work. However, one reason why this has not been considered yet is that calling back to the parent space is only intended to be used under special circumstances that require synchronised access to resources which are shared by all the isolates.

All calls back to the parent space are serialised, so they become a bottleneck for a parallel application. You would get much better throughput by copying the code in question into the isolate, so that all computation can proceed in parallel.

I hope this makes sense, and that it is possible for you to make this change. If not, please tell us more about the structure of your application!

[mfleming]

Hi Morton,

Thanks for the response. Hope your travel went well (New York?) I've spent more time with the document Parallel Language Features as well as an example Mandelbrot program from Brian Becker to get a better understanding of how the parallel execution system works. My inference is that you spawn a separate APL interpreter process connected to the main process by the RPC protocol. It all sounds much like distributing an Erlang application across multiple nodes.

My application is something of the inverse of the Madelbrot problem that applies a simple position independent function across an array of points. I've implemented a ray tracer program that applies a very complex independent computation across a canvas of pixels. So, rather than executing a single function with arguments and returning a result, what I would really need to do is execute a function within a namespace containing many dependent functions. I would pass a scene description and canvas coordinate to the main function and it would compute intersections, normals, reflections, etc. and return the color at the specified pixel.

I've done much to optimize computations after profiling the application (and there's probably a lot more could be done), so now I'd like to split up the rendering task and parcel things out to multiple CPUs. I've played around with a workspace with simple functions like in the first post. Callbacks to the main process obviously won't work. I've tried using a LOAD command in the function I'm passing to the isolate, but I usually end up killing the APL interpreter...

I guess I'm asking for a means to manipulate the remote APL process and set up its environment before sending it a function. You can have a look at the code itself on github, but I think there is little in the way of single functions without dependencies that can be distributed.

https://github.com/mafleming/RayTracerChallenge

Hopefully I've got the right mental model of the isolate implementation!
~Mark

[Morten|Dyalog]

Hope your travel went well (New York?)

Now in transit at Heathrow after Boston, Stamford, New York City, a couple of places in New Jersey, Seattle and Vancouver. Two nights in my own bed and then Stuttgart (and the week after, Milan). A little bit crazy at the moment, but all good meetings!

Did you read section 9.2 on "Serving isolates from your own application workspace"? http://docs.dyalog.com/17.0/Parallel%20 ... atures.pdf. It contains instructions on how to set up a workspace that will be loaded by all the isolates that you launch. You need to copy the isolate namespace into your application and make sure that the latent expression runs some code that will cause it to start serving things up.

If that isn't attractive, you should be able to use #.⎕CY from inside your isolate to copy anything you like into the root namespace of each isolate process, or indeed use any other mechanism to bring code in. You may not simply )LOAD another workspace, that will saw off the branch you are sitting on and close the TCP socket.

I'll try to take a look at your code over the next few days but I may not get there, so please keep asking questions if you need further assistance.

[mfleming]

You may not simply )LOAD another workspace, that will saw off the branch you are sitting on and close the TCP socket.

That analogy is quite enlightening!

I hadn't reached the Further Reading section because I was still struggling with earlier sections. So, I gave the sample program a try but it throws a VALUE ERROR where it attempts to do a short delay in each isolate. Perhaps there is a context setting I am missing?

The ⎕CY approach does work though in my simple foo bar example. Each function seems to be returning null however. There is a second thread executing a Wait function so it appears communication is not quite right. I've modified the function being passed to ll.Each as

      foo←{
     #.⎕CY 'workspace path and name'
     ⍺ bar ⍵
 }

I'll have a further look at PEACH and other samples.
~Mark

[petermsiegel]

Do you want to do:

Code: Select all

  foo←{
     _←#.⎕CY 'workspace path and name'   ⍝ ignore null result and keep going...
     ⍺ bar ⍵
 }

so the copy (⎕CY...) isn't the last thing executed in foo (returning null)?

[mfleming]

Ah, good catch! Should have been obvious I was exiting after loading the workspace (no wonder it seemed to work but return nulls). Unfortunately, I'm back to the usual VALUE ERROR in run1iso when it tries to execute bar in the new workspace environment.

Back to the manual...

[Morten|Dyalog]

Note that if you do a #.⎕CY, you will copy bar into # in the isolate workspace, this is NOT the default execution environment for an isolate, which is an anonymous namespace inside that process. Thus, you need to refer to #.bar, or set ⎕PATH up inside the isolate:

Code: Select all

      is←isolate.New ''
      is.(⍕⎕THIS) ⍝ return "name" of isolate namespace
#.isolate.ynys.[Namespace].[Namespace]
      is.('queens' #.⎕CY 'C:\Program Files\Dyalog\Dyalog APL-64 17.0 Unicode\ws\dfns')
      is.(#.queens 4) ⍝ explicit reference to queens in #
 · ⍟ · ·
 · · · ⍟
 ⍟ · · ·
 · · ⍟ ·
      is.(⎕PATH←'#')
#
      is.(queens 4) ⍝ ⎕PATH means the # is not required
 · ⍟ · ·
 · · · ⍟
 ⍟ · · ·
 · · ⍟ ·

A couple of gotchas:

Code: Select all

      is.⎕THIS
DOMAIN ERROR
      is.⎕THIS
      ∧


This expression fails because ⎕THIS is a reference, and you cannot move references between processes. You can however return the result of is.(⍕⎕THIS), which is a character vector.

Code: Select all

      is.queens 4
FUTURE ERROR: 2: SYNTAX ERROR: iEvaluate[13] (,⍕(⍕rc),': ',(0⊃res),{(⍵∨.≠' ')/': ',⍵}1⊃res,'' '')iSpace.qsignal rc
      is.queens 4
      ∧


This fails because we need to inspect the valence etc of a function that is called remotely, and ⎕PATH doesn't allow this inspection to happen. This is the same reason why your original call to #.bar in the host workspace could not be executed. However, if you write is.(queens 4), the entire expression is shipped off to be executed in the isolate, and this allows ⎕PATH to kick in.

There is only a small amount of magic involved in making calls to and from isolates. Once they are set up, they are just "normal" namespaces inside the host process.

[mfleming]

Creating and communicating directly with isolates, as your example and Brian's Mandelbrot program do, seems easier to understand. With the ll.Each example on the other hand, I modified the function being passed to the isolates as follows

      ∇Z←X foo Y
 #.⎕CY'C:\Users\Mark\Jupyter\threads'
 #.⎕PATH←'#'
 Z←X bar Y
∇

That worked the first time, returning the expected results, but left four processes each running at about 20% CPU utilization. The second time I ran main I got the error message

ISOLATE: Connection to localhost:7052 failed: 100 TIMEOUT

The processes did not response to an "isolate.Reset 0" and had to be killed.

If isolates are much like namespaces, then I need a better understanding of namespace manipulation as objects. Meanwhile, I will create a single isolate, pass things to it and then two isolates, and so forth. Build things up until the toy example works, at which point I can apply it to the ray tracer application. The application should benefit a great deal from threading since there is a lot of processing associated with each datum returned.

No doubt I'll kill more processes as I go along!

[Morten|Dyalog]

That is extremely strange, I can't think of an explanation for your observations. Which version of APL are you using?