Will (data,0)[;fields⍳field] create a copy in memory?

General APL language issues

Will (data,0)[;fields⍳field] create a copy in memory?

Postby OSRK on Mon Mar 23, 2020 12:29 pm

Will (data,0)[;fields⍳field] create a copy of data with 0 column added in memory?
User avatar
OSRK
 
Posts: 12
Joined: Tue Dec 27, 2016 8:48 am

Re: Will (data,0)[;fields⍳field] create a copy in memory?

Postby Morten|Dyalog on Tue Mar 24, 2020 8:15 am

Yes it will. If you have a lot of columns and performance is a consideration, you will want to work around it:

Code: Select all
      mat←?1e6 7⍴0 ⍝ 1 million rows, seven columns
      fields←⍳7
      field←99
      )copy dfns cmpx
...\ws\dfns.dws saved Mon Mar 16 19:59:36 2020
      cmpx '(mat,0)[;fields⍳field]' '(field∊fields)×mat[;(≢fields)⌊fields⍳field]' '{field∊fields:mat[;fields⍳⍵] ⋄ (≢mat)⍴0}field'
  (mat,0)[;fields⍳field]                       → 2.9E¯2 |    0% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕
  (field∊fields)×mat[;(≢fields)⌊fields⍳field]   → 5.9E¯3 |  -80% ⎕⎕⎕⎕         
  {⍵∊fields:mat[;fields⍳⍵] ⋄ (≢mat)⍴0}field → 7.8E¯6     | -100%                                         

 
User avatar
Morten|Dyalog
 
Posts: 451
Joined: Tue Sep 09, 2008 3:52 pm

Re: Will (data,0)[;fields⍳field] create a copy in memory?

Postby Roger|Dyalog on Thu Mar 26, 2020 7:13 pm

A good rule-of-thumb which answers the question is that outside of idioms, there is no lookahead for APL functions. In the phrase (data,0)[;fields⍳field] there are two functions, (data,0) and x[;y]. The (data,0) is done first, and then the indexing. At the time that it does the (data,0) the interpreter doesn't know that you are going to do indexing, and would have creating a matrix with a trailing 0 column. (Moreover, it doesn't know that it can do the optimizations in Morten's message.)

One can imagine that (x,sentinel)[y] or (x⍪sentinel)[y] (with 0 or 1 semicolons between the brackets) should be recognized as an idiom. Something to think about.

To see the list of existing idioms, in the menu bar of your Dyalog session, click Help → Language Help, then search for "idiom list".
Roger|Dyalog
 
Posts: 238
Joined: Thu Jul 28, 2011 10:53 am

Re: Will (data,0)[;fields⍳field] create a copy in memory?

Postby Fiona|Dyalog on Fri Mar 27, 2020 7:13 am

An alternative way to see all the existing idioms is to look at the cheat sheet (PDF): http://docs.dyalog.com/17.1/CheatSheet%20-%20Idioms.pdf
User avatar
Fiona|Dyalog
 
Posts: 68
Joined: Mon Apr 22, 2013 12:59 pm


Return to Language

Who is online

Users browsing this forum: No registered users and 1 guest