Generators and Iterators a la Python

Writing and using Classes in Dyalog APL

Generators and Iterators a la Python

Postby petermsiegel on Tue Nov 27, 2012 1:32 am

Anyone thinking about Python-like generators and iterators; generators can be thought of as "lazy" lists of mappings (simple or complex) of possibly infinite length.

The key innovation needed would be for APL built-in functions and operators, when faced with a user-defined class object, to expect and call certain class members, e.g. myclass.next, as well as to respond to certain signals (modeling Python StopIteration). While there are many elegant uses of generators and iterators we just don't need in APL, there are some uses already in place in an ad hoc fashion -- note how selection works today with certain default class methods (where it lazily waits before calling the method until it knows which object components it needs).

The <yield> keyword in Python (which allows generators to be programmed as if regular (co-)functions), yielding a value, while maintaining the current state, rather then returning a value. It can be simulated using threads, but requires different user functions to stay out of each other's way (e.g. in token management or using local variables to synchronize).

See:http://www.python.org/dev/peps/pep-0255/ (also, less elegantly, in many other languages).

Seems easy enough! Devils in the details.
petermsiegel
 
Posts: 160
Joined: Thu Nov 11, 2010 11:04 pm

Re: Generators and Iterators a la Python

Postby JohnS|Dyalog on Tue Nov 27, 2012 9:47 am

While nowhere near as neat as primitive language constructs, some of this functionality may already be modelled using namespaces, bound as operands or left arguments, to maintain persistent local state. See:
Function memoization: http://dfns.dyalog.com/n_memo.htm
Editor undo/redo stacks: http://dfns.dyalog.com/n_UndoRedo.htm
and this experimental version of dyalog, which implemented closures:
Closures: http://dfns.dyalog.com/downloads/fre.pdf
JohnS|Dyalog
 

Re: Generators and Iterators a la Python

Postby petermsiegel on Mon Dec 03, 2012 11:20 pm

I had seen the closures article, which was spot on; closures would be just the tool needed. But they aren't part of the product, right?
petermsiegel
 
Posts: 160
Joined: Thu Nov 11, 2010 11:04 pm

Re: Generators and Iterators a la Python

Postby JohnS|Dyalog on Tue Dec 04, 2012 8:58 am

Closures are not part of the product. You can model persistent local state by binding a namespace as left argument/operand:

Code: Select all
      next←(⎕ns'')∘{                    ⍝ space bound as left arg.
          0=⍵:⍺.n←0                     ⍝ next 0: reset number stream.
          (⍳⍺.n+←⍵)+⍺.n                 ⍝ next ⍵: return next ⍵ numbers.
      }

      next 0                            ⍝ reset count.

      next 1                            ⍝ next number.
1
      next 1                            ⍝ next number.
2
      next 2                            ⍝ next two numbers.
3 4
      next 1                            ⍝ next number.
5
      next 0                            ⍝ reset count.

      next 3                            ⍝ next three numbers,
1 2 3

See http://dfns.dyalog.com/n_memo.htm for more examples.
John.
JohnS|Dyalog
 

Re: Generators and Iterators a la Python

Postby David Lamkins on Tue Dec 04, 2012 8:24 pm

That's a neat technique.

Is there a reason I can't fix the 'next' function in Dyalog 13.1 for Linux?
David Lamkins
 
Posts: 21
Joined: Sun Aug 26, 2012 7:08 am

Re: Generators and Iterators a la Python

Postby David Lamkins on Tue Dec 04, 2012 8:34 pm

BTW, the technique works for me if I break it down into two functions, like this:


Code: Select all
 next∆←{
     0=⍵:⍺.n←0
     (⍳⍺.n+←⍵)+⍺.n
 }
 next←(⎕ns'')∘next∆                                                                             


It seems that my Linux version of Dyalog can't parse composition as part of a d-function definition.
David Lamkins
 
Posts: 21
Joined: Sun Aug 26, 2012 7:08 am

Re: Generators and Iterators a la Python

Postby Vince|Dyalog on Wed Dec 05, 2012 9:30 am

Hi David,

Our session doesn't allow you to join together multiple lines into one action. But, you can get this derived function next if you use diamonds and type it on one line in the session like this:

Code: Select all
next←(⎕ns'')∘{ 0=⍵:⍺.n←0 ⋄ (⍳⍺.n+←⍵)+⍺.n}


Regards,

Vince
Vince|Dyalog
 
Posts: 444
Joined: Wed Oct 01, 2008 9:39 am

Re: Generators and Iterators a la Python

Postby JohnS|Dyalog on Wed Dec 05, 2012 10:46 am

Apologies, I forgot that I was using a short-hand. Vince's one-liner-with-diamonds is good for short functions and your 2-stage definition is better for larger ones.

Note that you can also define next as an operator, which leaves the left argument of the resulting derived function free.

Code: Select all
      next∆←{⍺←0          ⍝ default: no reset:
         ⍺:⍺⍺.n←⍵-⍳1      ⍝ reset sequence to ⍵.
         (⍳⍺⍺.n+←⍵)+⍺⍺.n  ⍝ next ⍵ numbers.
      }

      next ← (⎕ns'')next∆  ⍝ NB: no '∘'

      1 next 0   ⍝ reset sequence to 0.

      next 2     ⍝ next 2 numbers
0 1
      next 0     ⍝ next 0 numbers

      next 3     ⍝ next 3 numbers
2 3 4
      1 next 42  ⍝ reset sequence to 42

      next 2     ⍝ next 2 numbers
42 43

NB: In the above, the multi-line D-fn may not be entered directly into the session.
Use )ed next∆ to define it.
JohnS|Dyalog
 

Re: Generators and Iterators a la Python

Postby David Lamkins on Wed Dec 05, 2012 6:49 pm

Thank you, John and Vince.

Your replies had me scratching my head for a moment. I know that dfns must be defined on one line in the session. I tried that this morning; of course it works.

Yesterday, though, I was attempting to define 'next' in an editor. This (your original version using the namespace composed with the function) gives me a "Can't Fix" message in the mode line regardless of whether the definition is on one line or not.

It seems, therefore, that the editor and the session have different syntax requirements for dfns. Is this expected?
David Lamkins
 
Posts: 21
Joined: Sun Aug 26, 2012 7:08 am

Re: Generators and Iterators a la Python

Postby JohnS|Dyalog on Thu Dec 06, 2012 8:28 am

It seems, therefore, that the editor and the session have different syntax requirements for dfns. Is this expected?

Yes, in this case, the session is being used to name the result of the evaluation of an expression:

      half ← ÷2              ⍝ naming result of function applied to array argument
sum ← +/ ⍝ naming result of operator applied to function operand
next ← (⎕ns'')∘{...} ⍝ naming result of operator applied to operands

The editor can define only the components (arrays, functions, operators) of such expressions.

John

PS. great advances have often arisen from questions such as these.
JohnS|Dyalog
 

Next

Return to Object Oriented Programming

Who is online

Users browsing this forum: No registered users and 1 guest