Need to convert APL Nested Array into JSON string

APL-related discussions - a stream of APL consciousness.
Not sure where to start a discussion ? Here's the place to be
Forum rules
This forum is for discussing APL-related issues. If you think that the subject is off-topic, then the Chat forum is probably a better place for your thoughts !

Need to convert APL Nested Array into JSON string

Postby woody on Sat Nov 07, 2020 4:11 am

Greetings!

I have a nested array that I would like to convert to JSON.

Header row
data row 1
data row 2
(etc.)

      MAT←1 3⍴   'Name'  'Age'  'Zipcode'  
MAT←MAT,[1]'Bob' '21' '30102'
MAT←MAT,[1]'Sally' '32' '43001'

]disp MAT
┌→────┬───┬───────┐
↓Name │Age│Zipcode│
├────→┼──→┼──────→┤
│Bob │21 │30102 │
├────→┼──→┼──────→┤
│Sally│32 │43001 │
└────→┴──→┴──────→┘


My objective is to convert the APL array into a JSON string:

'[{"Name":"Bob","Age":"21","Zipcode":"30102"},{"Name":"Sally","Age":"32","Zipcode":"43001"}]'

And... the inverse as well ..
from JSON string back to nested array.

I can force all numbers to be text via ⍕ converting 21 to '21' inside MAT ahead of time.
I would like to work with all text strings in all cells.

Thoughts?

Sincere thanks!

//W
Woodley Butler
Automatonics, Inc.
"Find your head in the APL Cloud"
http://www.APLcloud.com
User avatar
woody
 
Posts: 123
Joined: Tue Dec 28, 2010 12:54 am
Location: Atlanta, Georgia USA

Re: Need to convert APL Nested Array into JSON string

Postby Phil Last on Sat Nov 07, 2020 11:20 am

Look at ⎕JSON. In particular the HighRank option:

      ⎕JSON⍠'HighRank' 'Split'⊢MAT
[["Name","Age","Zipcode"],["Bob","21","30102"],["Sally","32","43001"]]

But with my limited investigation the inverse unfortunately seems not to have been implemented.
User avatar
Phil Last
 
Posts: 553
Joined: Thu Jun 18, 2009 6:29 pm

Re: Need to convert APL Nested Array into JSON string

Postby Phil Last on Sat Nov 07, 2020 11:28 am

With a small cover at either end you could store a two item list of shape and ravel but you'd have to be a bit cleverer if any item could itself be of higher rank.

Also JSON doesn't understand scalar chars. So 'Arthur' 'C' 'Clarke' would come back as 'Arthur'(,'C')'Clarke' after a round trip.
User avatar
Phil Last
 
Posts: 553
Joined: Thu Jun 18, 2009 6:29 pm

Re: Need to convert APL Nested Array into JSON string

Postby Morten|Dyalog on Sat Nov 07, 2020 11:40 am

Phil Last wrote:But with my limited investigation the inverse unfortunately seems not to have been implemented.


The reason for this is that, while you know any outbound matrix MUST be split in order to be represented in JSON, there is no easy way to decide whether an incoming list of lists should be mixed into a matrix, or the intention really was to transmit a vector of vectors.

I think what Woodley is looking for is a proposed extension that we are considering, where each row in a table is converted into an object, with each value is named using the column name. That might be easier to make invertible with some kind of option.

This might still be a candidate for v19.0, if anyone else thinks this would be useful, speak up and help us decide that the time has come to do it!
User avatar
Morten|Dyalog
 
Posts: 389
Joined: Tue Sep 09, 2008 3:52 pm

Re: Need to convert APL Nested Array into JSON string

Postby Phil Last on Sat Nov 07, 2020 11:51 am

My attempted implication was that as export requires the "HighRank" option it could flag the output such that import with the same option could take it into account and rebuild the array.
User avatar
Phil Last
 
Posts: 553
Joined: Thu Jun 18, 2009 6:29 pm

Re: Need to convert APL Nested Array into JSON string

Postby Morten|Dyalog on Sat Nov 07, 2020 1:01 pm

Adding some form of extension to JSON notation to mark higher rank arrays was considered, but we decide to push ahead without it because:

  • There was a real and urgent need to make it easier to generate JSON from existing APL data, which often contains matrices, for consumption by external components like JavaScript controls - without having to transform the APL data first.
  • External (non-APL) components will never return data in our extended format, and would presumably choke on it if we generated it.
  • When communicating from one APL system to another, there are more efficient alternatives to JSON.
We did not want to delay providing a useful feature while discussing JSON extensions. That doesn't mean we cannot adopt such an extension in the future, of course.
User avatar
Morten|Dyalog
 
Posts: 389
Joined: Tue Sep 09, 2008 3:52 pm

Re: Need to convert APL Nested Array into JSON string

Postby Phil Last on Sat Nov 07, 2020 5:21 pm

All three points taken. All reservations withdrawn.
User avatar
Phil Last
 
Posts: 553
Joined: Thu Jun 18, 2009 6:29 pm

Re: Need to convert APL Nested Array into JSON string

Postby Adam|Dyalog on Sun Nov 08, 2020 7:27 am

This should do it:
      {⎕JSON(1⌷⍵){r←⎕NS⍬ ⋄ r⊣r⍎⍕⍺'←⍵'}⍤1⊢1↓⍵}MAT
[{"Age":"21","Name":"Bob","Zipcode":"30102"},{"Age":"32","Name":"Sally","Zipcode":"43001"}]


We're isolating the column descriptions and pairing that row up with remaining rows (thus using the header row repeatedly). Every time we do such a pairing, we create a namespace, then create an expression that distributes the values to the corresponding names, execute that expression in the new namespace and return the namespace. Finally, the whole things gets converted to ⎕JSON.
User avatar
Adam|Dyalog
 
Posts: 84
Joined: Thu Jun 25, 2015 1:13 pm

Re: Need to convert APL Nested Array into JSON string

Postby Morten|Dyalog on Sun Nov 08, 2020 8:08 am

Couldn't resist a little Sunday morning exercise. The code isn't very robust, but it works on the given sample.

Code: Select all
 JSONTableMK←{
   cols←',"'∘,¨⍵[1;],¨⊂'":'
   char←⍸0=10|⎕DR¨⊃¨(data←1↓⍵)[1;] ⍝ Assume 1st row can be used to determine type
   data[;char]←'"',¨data[;char],¨'"'
   '[',']',⍨¯1↓∊'{',¨1↓¨(∊¨↓cols(,¨⍤1)⍕¨data),¨⊂'},'
 }

I had to reorder MAT[;1 2] to get the columns in alphabetical order, to get the same result when testing. Alas, namespaces are a bit heavy for this:

Code: Select all
      cmpx 'JSONTableAB MAT' 'JSONTableMK MAT'
  JSONTableAB MAT → 3.8E¯5 |   0% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕
  JSONTableMK MAT → 2.3E¯5 | -40% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕               

      cmpx 'JSONTableAB MAT' 'JSONTableMK MAT' ⍝ With 100 rows
  JSONTableAB MAT → 1.8E¯3 |   0% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕
  JSONTableMK MAT → 4.3E¯4 | -76% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕   

      cmpx 'JSONTableAB MAT' 'JSONTableMK MAT' ⍝ With 1000
  JSONTableAB MAT → 2.0E¯2 |   0% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕
  JSONTableMK MAT → 3.1E¯3 | -85% ⎕⎕⎕⎕⎕⎕                                 
                           
User avatar
Morten|Dyalog
 
Posts: 389
Joined: Tue Sep 09, 2008 3:52 pm

Re: Need to convert APL Nested Array into JSON string

Postby woody on Tue Nov 10, 2020 1:35 am

Good stuff!

Works like a charm...


      ∇ Z←JSONTableMK MAT;cols;char;data
[1] cols←',"'∘,¨MAT[1;],¨⊂'":'
[2] char←⍸0=10|⎕DR¨⊃¨(data←1↓MAT)[1;]
[3] ⍝ Assume 1st row can be used to determine type
[4] data[;char]←'"',¨data[;char],¨'"'
[5] Z←'[',']',⍨¯1↓∊'{',¨1↓¨(∊¨↓cols(,¨⍤1)⍕¨data),¨⊂'},'
[6]


Cheers
Woodley Butler
Automatonics, Inc.
"Find your head in the APL Cloud"
http://www.APLcloud.com
User avatar
woody
 
Posts: 123
Joined: Tue Dec 28, 2010 12:54 am
Location: Atlanta, Georgia USA


Return to APL Chat

Who is online

Users browsing this forum: No registered users and 1 guest