Conga in the Key of D

[paulmansour]

I'm fooling around with Conga 2.0, as client only, to connect to a web service (well I'm not sure its a officially a web service, but it accepts http with some XML and sends back the same). Anyway, my starting point was the sample Samples.HTTPGet function included in the Conga namespace. From this I extracted a little D function to do the receiving part:

Code: Select all

Receive←{
     ⎕IO←0 ⋄ ⎕ML←3
     ⍺←''
     r←DRC.Wait ⍵ 20000 
     0≠↑r:(↑r)(1⊃r)''                   
     'BlockLast'≡2⊃r:0 ''(⍺,3⊃r)             
     'Error'≡2⊃r:1(3⊃r)''               
     (⍺,3⊃r)∇ ⍵                         
 }


The right arg is simply the connection name. The result is three items: a return code, an error message, and then the received data, inlcuding the header.

In the HTTPGet function, there is additional checking to see if the content length has been reached, which requires partially processing the header as you loop around the Wait function.

My questions are:

1) Is one guaranteed to get a "BlockLast" event?
2) If you wait to terminate until the BlockLast, will your data received perhaps be longer than the specified content lenght in the header?
3) Are you guaranteed to get at least the entire header in the first call to Wait?

The above function appears to work, but I'm sure it's a bit simplistic. Any comments or pointers would be appreciated.

[Bjørn|Dyalog]

1) Is one guaranteed to get a "BlockLast" event?

No. When the connection is closed, Conga returns the content of the buffer received and marked it as BlockLast.


2) If you wait to terminate until the BlockLast, will your data received perhaps be longer than the specified content length in the header?

If you wait until the BlockLast you will be waiting until the connection is closed, and that can happened long time after you have received the last block of you data, the Content-Length is part of the HTTP protocol and will tell you when you have received your data.


3) Are you guaranteed to get at least the entire header in the first call to Wait?

No, that depends on how busy the server is, and the size of the max network packages. From empiric data I have not seen the header devided into multible block.

[paulmansour]

Bjørn,

Thanks for the info. I have revamped things, taking into consideration all the issues above, adding two sub functions, one for the header and one for any remaining content:

Code: Select all

Receive←{                       ⍝ Receive HTTP
     h c←ReceiveHeader ⍵         ⍝ Get header, content fragment
     nv←1↓¨(~(⎕UCS 13 10)⍷h)⊂h   ⍝ Name/Value pairs
     n v←↓⍉⊃2↑¨{(~<\⍵=':')⊂⍵}¨nv ⍝ Split 'em up
     i←n⍳⊂'Content-Length'       ⍝ Find length
     l←↑1⊃⎕VFI i⊃v               ⍝ Numeric
     l≤⍴c:c{⍺}DRC.Close ⍵        ⍝ Already Done
     c←l c ReceiveContent ⍵      ⍝ Get the content.
     c{⍺}DRC.Close ⍵             ⍝
 }

ReceiveHeader←{               ⍝ Receive header only
     ⍺←''                      ⍝ Header
     r←DRC.Wait ⍵ 20000        ⍝ Wait
     en←799                    ⍝ Error number
     0≠↑r:(1⊃r)⎕SIGNAL en      ⍝ Error
     e d←r[2 3]                ⍝ Event, Got some data
     e≡'Error':d ⎕SIGNAL en    ⍝ Another Error
     h←⍺,d                     ⍝ Accumulate
     b←(4⍴⎕UCS 13 10)⍷h        ⍝ Header break
     1∊b:0 3↓¨(~<\b)⊂h         ⍝ end of header found
     'BlockLast'≡e:h''         ⍝ Done (or error)
     h ∇ ⍵                     ⍝ Read more data
 }

ReceiveContent←{            ⍝ Receive content
     l c←⍺                   ⍝ Length, starting content fragment
     r←DRC.Wait ⍵ 20000      ⍝ Wait for data
     en←799                  ⍝ Error number
     0≠↑r:(1⊃r)⎕SIGNAL en    ⍝ Timeout or other early error
     e d←r[2 3]              ⍝ Event, Data
     e≡'Error':d ⎕SIGNAL en  ⍝ Error
     c,←d                    ⍝ Accumulate received bytes
     l≤⍴c:c                  ⍝ Content length reached
     e≡'BlockLast':c         ⍝ Socket closed
     l c ∇ ⍵                 ⍝ Read more
 }


It seems to me that this functionality would be a fairly common requirement for dealing with any web service. I didn't see any stand-alone receive functionality in the Conga samples. The only place I found it was embedded in the HTTPGet function, so I had to extract it anyway, and figured I'd do it in D. Did I miss something in the samples?

Why wouldn't DCR have a built in HttpRecieve function? Is there just too much variability in what can come back and what people would want in the result?