:With oddities

[Tomas Gustafsson]

Segment below:

Code: Select all

 h_receiveasync arg;a;hdr;ep;eps;ind;ns;pBOOL;pFLO8;pINT2;pINT4;DATA;DESC;EP;EPs;TBOOL;TFLO8;TINT4
⍝ Receive the data and re-start the async BeginReceive method

 {}#.SOCKET.socket.BeginReceive #.SOCKET.hand arg.AsyncState ⍝ Spawn a new BeginReceive right away
 ep←#.ByRef.New⊂#.Net.IPEndPoint.New(#.Net.IPAddress.IPv6Any)0
 DATA←#.SOCKET.socket.EndReceive arg ep
 EP←ep.Value                                             ⍝ EP now holds senders ip and port

 :With #.SOCKET                                          ⍝ This is an ordinary Dyalog APL namespace
     EPs←EP.ToString
     .... 

VALUE ERROR, because EP is no longer available, even though it's localized in the fn header (the ToString method does exist ofc).

It has been convenient to use local variables, regardless of the :With [namespace] in effect. As long as they are assigned, also outside the :With segment. But the System.Net objects don't seem to follow that rule. The ByRef doesn't particularly cause this, if the first fn line was :With #.SOCKET, i couldn't access the rarg (named "arg"). This phenomen is consistent all over the ws.

If i replace the last lines with:

Code: Select all

 #.SOCKET.EPs←EP.ToString
     .... 

... it works. Usually i've been able to get over it by using less :With's, but when the code basically works in #.SOCKET, it's a lot of extra typing (and work for the interpreter?).

Ideas? Thx / Tomas

Interestingly, this is a super-fast way to create APL threads. Each incoming packet fails, the next incoming calls the same fn that fails again, but in a new thread since the previous isn't ready yet (failed), etc. Doing this multiple times per second quickly fills my home with Dyalog APL threads :)

13.2.16716.0 32 Classic, Jan 30, 2013, .net 4.5, Win7

[Vince|Dyalog]

Hi Tomas,

Perhaps you can refer to EP as #.EP or ##.EP within the :with?

Regards,

Vince

[Tomas Gustafsson]

Hey Vince! Here's part of what i wrote Morten a moment ago:

My whole programming strategy is based on fixed (saved with the ws) namespaces and temporary namespaces (created at app start, deleted at app end). The temporary ones act as storgage locations for all persisting data, that you understand exists in a looping application (next loop works on same data). At most, they may carry some dynamically created dfn, but otherwise data only – lots of identifiers. The amount of dynamic data is so large in this kind of app, that it must be in hard hands. So i never )save any temporary data – it’s always cleared out, clean exit. It’s almost clinical and has to be, since i may re-start the app from session 100 times. Each time must happen from a “clear environment”.

At most, i counted to 17 :With :End segments in the biggest one, the game loop fn itself.

Typically, a fn starts with

:With #.SOCKET, or #.WATER, #.BOAT, #.GEO and 20-30 others.

Ie. i keep the function namespaces and the data the work on separated. Thanks to that, i never see variables hanging around in #._Water, since #._Water.foo executes in #.WATER. Often the interconnect, so foo may :With-engage several of these #.[UPPERCASE].[MORELEVELS] namespaces

I tell you, this is the only way :-). I’d be lost if trying to insert namespace identifiers in front of all (namespace level) globals. Hope JD brings some light on this.

I was able to override this one, re-arranging a bit. But i really hope that JohnD will shed some light on the logic behind :With, and maybe also on it's performance/overhead. Thanks!

[JohnS|Dyalog]

Hi Tomas,
At least part of your problem stems from the design of namespaces prior to the introduction of refs as first-class citizens of Dyalog.
Our original model for namespaces was as a tree-like parent/child hierarchy or a set of nested containers. In such a model, one is always either inside or outside a particular space but not both. With this model, the following code snippet does not make sense:

      ∇ foo;ns
[1]   'ns'⎕ns''      ⍝ local namespace
[2]   9=⎕nc'ns'      ⍝ we're _outside_ ns, so can see it.
[3]   :With ns       ⍝ now we're _inside_ ns,
[4]       0=⎕nc'ns'  ⍝ so can't see it.
[5]       :With ns   ⍝ and this is clearly crazy.
...

It would be as crazy as expecting the following analogy to work with an operating system file/directory hierarchy:

$ ls            # show directories
bin tmp usr
$ cd tmp    # now we're inside the tmp directory
$ ls            # this is clearly broken!
bin tmp usr

To avoid this paradox, we made the rule that local names would be visible from inside a :With block except if they had class 9 (namespace).

As you have discovered, it is usually possible to code around such problems with a little re-organisation.

Regarding the performance of :With - it does little more than switch a symbol-table pointer, together with some interpretative overhead. You can check for yourself by timing loops which do or don't include a :With struct.

The following experiment (using Dyalog V12.1 on a fairly old machine) shows that :With/:EndWith takes about the same time as, say, reversing an eighty-item floating point vector.

      disp ⎕cr¨'loopA' 'loopB' 'loopC'
┌→────────┬─────────┬─────────┐
│i←loopA i│i←loopB i│i←loopC i│
│         │:With ⎕SE│v←⌽v     │
│         │:EndWith │         │
│→0<i←i-1 ↓→0<i←i-1 ↓→0<i←i-1 ↓
└────────→┴────────→┴────────→┘
      )copy dfns time
u:\ws\dfns saved Mon Mar 04 12:55:30 2013

      v←.5+⍳80  ⍝ vector of floats

      loopA time 1e6   ⍝ empty loop
00.62
      loopB time 1e6   ⍝ :With loop
03.03
      loopC time 1e6   ⍝ reversing loop
03.64

John.

[Tomas Gustafsson]

Thank you very much John for this clarification.

My immediate reaction was that why couldn't a (.net) namespace be included in the :With domain, since they have clear property etc. names. But of course that would cater for oddities, since i can say

UdpSocket.IsItRainingToday <- 1

The namespaces are functional, are they native or .net, without restrictions on the identifiers. What you say is fair and logical.

Performance wise, i will now see over the code for "lazy" :With :End segments, and replace those with common dot-separation or alike.

But re that, i'd like to ask one more question about the performance: Without any extensive benchmarking, could you say if the :With overhead is proportional to the overall, general size of the function; ie. the avg number if identifiers, or likewise?

A :With-segment in a fn with 300 lines, 300 identifiers is more expensive that a :With in a fn with 5 lines and 5 identifiers?

That would provide some excellent general guidance on where to put in the biggest efforts in using or avoiding :With, which is otherwise highly useful to structure the code.

Thx! / Tomas

[JohnS|Dyalog]

Regarding the performance of :With - it does little more than switch a symbol-table pointer, together with some interpretative overhead.

Correction: :With "imports" the local names (except for those of class 9) of its containing function. This means that the cost of a :With is proportional to the number of these local names.

"Namespace dotting" does not incur this overhead, so while

      ns.(aa←bb)

may be inconvenient for larger tasks, it is quicker than

      :With ns
         aa←bb
      :End

if there are a significant number of local names.

This overhead for :With depends only on the number of local names, not on the number of lines or tokens in the containing function.

John.

[Tomas Gustafsson]

Thank you John!

      Info
    384 functions    
324,258 bytes of code
 12,049 lines of code

Comment bytes not counted. 250 of my 384 functions in the ws contain one or more :With.

245 hold a '{'

65 and 64 %. You have 0 and 100 %. :-)

[Nicolas|Dyalog]

      ∇ foo;ns
[1]   'ns'⎕ns''      ⍝ local namespace
[2]   9=⎕nc'ns'      ⍝ we're _outside_ ns, so can see it.
[3]   :With ns       ⍝ now we're _inside_ ns,
[4]       0=⎕nc'ns'  ⍝ so can't see it.
[5]       :With ns   ⍝ and this is clearly crazy.
[6]       :EndWith
[7]   :EndWith
∇

For people who reckon that line [5] is reasonable, you can still do it by hiding the ref into an array :

      ∇ foo;ns;var
[1]   'ns'⎕ns''        ⍝ local namespace
[2]   var←,ns          ⍝ put ref into array
[3]   9=⎕nc'ns'        ⍝ ns is a ref
[4]   2=⎕nc'var'       ⍝ var is a variable
[5]   :With ns         ⍝ now we're _inside_ ns,
[6]       0=⎕nc'ns'    ⍝ local refs have disappeared
[7]       2=⎕nc'var'   ⍝ but not variables
[8]       :With ns     ⍝ this fails with VALUE ERROR
[9]       :EndWith
[10]      :With var[1] ⍝ but this works !
[11]      :EndWith
[12]  :EndWith
∇

Cheers,

Nic.