Need to convert APL Nested Array into JSON string

[woody]

Greetings!

I have a nested array that I would like to convert to JSON.

Header row
data row 1
data row 2
(etc.)

      MAT←1 3⍴   'Name'  'Age'  'Zipcode'  
MAT←MAT,[1]'Bob'   '21'   '30102'
MAT←MAT,[1]'Sally' '32'   '43001'

      ]disp MAT   
┌→────┬───┬───────┐
↓Name │Age│Zipcode│
├────→┼──→┼──────→┤
│Bob  │21 │30102  │
├────→┼──→┼──────→┤
│Sally│32 │43001  │
└────→┴──→┴──────→┘

My objective is to convert the APL array into a JSON string:

'[{"Name":"Bob","Age":"21","Zipcode":"30102"},{"Name":"Sally","Age":"32","Zipcode":"43001"}]'

And... the inverse as well ..
from JSON string back to nested array.

I can force all numbers to be text via ⍕ converting 21 to '21' inside MAT ahead of time.
I would like to work with all text strings in all cells.

Thoughts?

Sincere thanks!

//W

[Phil Last]

Look at ⎕JSON. In particular the HighRank option:

      ⎕JSON⍠'HighRank' 'Split'⊢MAT
[["Name","Age","Zipcode"],["Bob","21","30102"],["Sally","32","43001"]]

But with my limited investigation the inverse unfortunately seems not to have been implemented.

[Phil Last]

With a small cover at either end you could store a two item list of shape and ravel but you'd have to be a bit cleverer if any item could itself be of higher rank.

Also JSON doesn't understand scalar chars. So 'Arthur' 'C' 'Clarke' would come back as 'Arthur'(,'C')'Clarke' after a round trip.

[Morten|Dyalog]

But with my limited investigation the inverse unfortunately seems not to have been implemented.

The reason for this is that, while you know any outbound matrix MUST be split in order to be represented in JSON, there is no easy way to decide whether an incoming list of lists should be mixed into a matrix, or the intention really was to transmit a vector of vectors.

I think what Woodley is looking for is a proposed extension that we are considering, where each row in a table is converted into an object, with each value is named using the column name. That might be easier to make invertible with some kind of option.

This might still be a candidate for v19.0, if anyone else thinks this would be useful, speak up and help us decide that the time has come to do it!

[Phil Last]

My attempted implication was that as export requires the "HighRank" option it could flag the output such that import with the same option could take it into account and rebuild the array.

[Morten|Dyalog]

Adding some form of extension to JSON notation to mark higher rank arrays was considered, but we decide to push ahead without it because:

• There was a real and urgent need to make it easier to generate JSON from existing APL data, which often contains matrices, for consumption by external components like JavaScript controls - without having to transform the APL data first.
• External (non-APL) components will never return data in our extended format, and would presumably choke on it if we generated it.
• When communicating from one APL system to another, there are more efficient alternatives to JSON.

We did not want to delay providing a useful feature while discussing JSON extensions. That doesn't mean we cannot adopt such an extension in the future, of course.

[Phil Last]

All three points taken. All reservations withdrawn.

[Adam|Dyalog]

This should do it:

      {⎕JSON(1⌷⍵){r←⎕NS⍬ ⋄ r⊣r⍎⍕⍺'←⍵'}⍤1⊢1↓⍵}MAT
[{"Age":"21","Name":"Bob","Zipcode":"30102"},{"Age":"32","Name":"Sally","Zipcode":"43001"}]

We're isolating the column descriptions and pairing that row up with remaining rows (thus using the header row repeatedly). Every time we do such a pairing, we create a namespace, then create an expression that distributes the values to the corresponding names, execute that expression in the new namespace and return the namespace. Finally, the whole things gets converted to ⎕JSON.

[Morten|Dyalog]

Couldn't resist a little Sunday morning exercise. The code isn't very robust, but it works on the given sample.

Code: Select all

 JSONTableMK←{
   cols←',"'∘,¨⍵[1;],¨⊂'":'
   char←⍸0=10|⎕DR¨⊃¨(data←1↓⍵)[1;] ⍝ Assume 1st row can be used to determine type
   data[;char]←'"',¨data[;char],¨'"'
   '[',']',⍨¯1↓∊'{',¨1↓¨(∊¨↓cols(,¨⍤1)⍕¨data),¨⊂'},'
 }


I had to reorder MAT[;1 2] to get the columns in alphabetical order, to get the same result when testing. Alas, namespaces are a bit heavy for this:

Code: Select all

      cmpx 'JSONTableAB MAT' 'JSONTableMK MAT'
  JSONTableAB MAT → 3.8E¯5 |   0% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕
  JSONTableMK MAT → 2.3E¯5 | -40% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕               

      cmpx 'JSONTableAB MAT' 'JSONTableMK MAT' ⍝ With 100 rows
  JSONTableAB MAT → 1.8E¯3 |   0% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕
  JSONTableMK MAT → 4.3E¯4 | -76% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕   

      cmpx 'JSONTableAB MAT' 'JSONTableMK MAT' ⍝ With 1000
  JSONTableAB MAT → 2.0E¯2 |   0% ⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕⎕
  JSONTableMK MAT → 3.1E¯3 | -85% ⎕⎕⎕⎕⎕⎕                                 
                           


[woody]

Good stuff!

Works like a charm...

      ∇ Z←JSONTableMK MAT;cols;char;data
[1]    cols←',"'∘,¨MAT[1;],¨⊂'":'
[2]    char←⍸0=10|⎕DR¨⊃¨(data←1↓MAT)[1;]
[3]   ⍝ Assume 1st row can be used to determine type
[4]    data[;char]←'"',¨data[;char],¨'"'
[5]    Z←'[',']',⍨¯1↓∊'{',¨1↓¨(∊¨↓cols(,¨⍤1)⍕¨data),¨⊂'},'
[6]
     ∇

Cheers