Reading streaming data from external Websocket connection
Forum rules
This forum is for discussing APL-related issues. If you think that the subject is off-topic, then the Chat forum is probably a better place for your thoughts !
This forum is for discussing APL-related issues. If you think that the subject is off-topic, then the Chat forum is probably a better place for your thoughts !
3 posts
• Page 1 of 1
Reading streaming data from external Websocket connection
Greetings.
I need a few pointers to get me going in the right direction regarding how to connect an APL session to read streaming stock market data from an external Websocket service.
My primary goal is to read and STORE the streaming stock market data ... perhaps in an APL component file. It's a lot of data ... and a single day of stock quotes for one Stock Symbol might be 100,000 records (100 bytes each).
Here's the HTTP steps to setup the streaming HTTP URL from the Stock Market data broker.
I know how to do this ... but I am not sure how to connect to the streaming data.
curl -X POST "https://api.tradier.com/v1/markets/events/session" \
-H 'Authorization: Bearer <TOKEN>' \
-H 'Accept: application/json'
Response (JSON)
{
"stream": {
"url": "https://stream.tradier.com/v1/markets/events",
"sessionid": "c8638963-a6d4-4fb9-9bc6-e25fbd8c60c3"
}
}
But, what Dyalog tool would I use to open a Websocket connection ?
And what might be an example of getting the streaming data ... and appending it to an APL component file.
Do I simply use HTTPGet ... and LOOP to extract BLOBS of data from the stream ?
How do I get the streaming data without missing something from the stream as I Loop ?
Thanks in advance for the help on this.
Cheers,
//W
I need a few pointers to get me going in the right direction regarding how to connect an APL session to read streaming stock market data from an external Websocket service.
My primary goal is to read and STORE the streaming stock market data ... perhaps in an APL component file. It's a lot of data ... and a single day of stock quotes for one Stock Symbol might be 100,000 records (100 bytes each).
Here's the HTTP steps to setup the streaming HTTP URL from the Stock Market data broker.
I know how to do this ... but I am not sure how to connect to the streaming data.
curl -X POST "https://api.tradier.com/v1/markets/events/session" \
-H 'Authorization: Bearer <TOKEN>' \
-H 'Accept: application/json'
Response (JSON)
{
"stream": {
"url": "https://stream.tradier.com/v1/markets/events",
"sessionid": "c8638963-a6d4-4fb9-9bc6-e25fbd8c60c3"
}
}
But, what Dyalog tool would I use to open a Websocket connection ?
And what might be an example of getting the streaming data ... and appending it to an APL component file.
Do I simply use HTTPGet ... and LOOP to extract BLOBS of data from the stream ?
How do I get the streaming data without missing something from the stream as I Loop ?
Thanks in advance for the help on this.
Cheers,
//W
-
woody - Posts: 144
- Joined: Tue Dec 28, 2010 12:54 am
- Location: Atlanta, Georgia USA
Re: Reading streaming data from external Websocket connectio
Hi Woodley,
I've taken a look at the online documentation for the streaming API for tradier.com. They are not using WebSocket protocol, but rather HTTP streaming which is achieved by leaving the socket open and using "chunked" transfer encoding.
HttpCommand supports chunked transfer encoding. However, it expects that the response from the server is a single response that's broken up into chunks of a more manageable size. That's not the case here; tradier.com is sending individual (stand alone) chunks in a stream.
To get streaming data from tradier.com, you'll need to write something that:
The logic is pretty straightforward and HttpCommand has code that could be cloned and modified to help accomplish this.
I hope this helps!
/Brian
I've taken a look at the online documentation for the streaming API for tradier.com. They are not using WebSocket protocol, but rather HTTP streaming which is achieved by leaving the socket open and using "chunked" transfer encoding.
HttpCommand supports chunked transfer encoding. However, it expects that the response from the server is a single response that's broken up into chunks of a more manageable size. That's not the case here; tradier.com is sending individual (stand alone) chunks in a stream.
To get streaming data from tradier.com, you'll need to write something that:
- Issues a POST request to get a streaming sessionid
- Issues a GET or POST request using that sessionid and whatever other parameters you need
- Sets up a listening loop in a separate thread to receive the chunks as the server pushes them out. Upon receipt of a chunk, your application code would process the data (probably in a new thread so as not to tie up the listening loop).
The logic is pretty straightforward and HttpCommand has code that could be cloned and modified to help accomplish this.
I hope this helps!
/Brian
-
Brian|Dyalog - Posts: 116
- Joined: Thu Nov 26, 2009 4:02 pm
- Location: West Henrietta, NY
Re: Reading streaming data from external Websocket connectio
Sincere thanks !
I'll give it a try.
Cheers,
//W
I'll give it a try.
Cheers,
//W
-
woody - Posts: 144
- Joined: Tue Dec 28, 2010 12:54 am
- Location: Atlanta, Georgia USA
3 posts
• Page 1 of 1
Who is online
Users browsing this forum: No registered users and 1 guest
Powered by phpBB © 2000, 2002, 2005, 2007 phpBB Group