Creating a WS under Program Control

[paulmansour]

Thanks all for the responses.

Stefano, thanks very much your suggestion regarding redirecting standard input. I tried it and it works! I tried it from the command line using a little text file with a few lines of code for input. I assume there is some way I can do this from another Dyalog session using ⎕CMD or .NET equivalent without even having a file for input... but I have not tried that yet.

For what I am looking to do, I wouldn't even need to fix a lot of code because I can assume a dyalog installation with certain installed user commands.

Very nice! A lot of possibilities here.

[StefanoLanzavecchia]

Sure can!

Here's a snippet in "C#" (it's Cake, but it's basically C#). You won't be able to copy it as-is because it depends on other code but it'll give you the main gist and you can read it as pseudocode:

Code: Select all

using (var process = new System.Diagnostics.Process {
   EnableRaisingEvents = true,
   StartInfo = new System.Diagnostics.ProcessStartInfo(exe, ps.Arguments.Render())
   {
      RedirectStandardInput = true,
      RedirectStandardOutput = true,
      RedirectStandardError = true,
      UseShellExecute = false
   }
   }) {
       Action<object, System.Diagnostics.DataReceivedEventArgs, string> actionWrite = (sender, e, type) =>
      {
         if(!string.IsNullOrWhiteSpace(e.Data)) {
            var msg = $"{type}> {e.Data}";
            if(e.Data.TrimStart().StartsWith("ERROR")) {
               ctx.Error(msg);
            } else {
               ctx.Information(msg);
            }
         }
      };
         
      process.ErrorDataReceived += (sender, e) => actionWrite(sender, e, "2");
      process.OutputDataReceived += (sender, e) => actionWrite(sender, e, "1");

      process.Start();
      process.BeginOutputReadLine();
      process.BeginErrorReadLine();

      var inputScript = ctx.FileReadText(bootstrap);
         
      ctx.Information($"====== Input script:{Environment.NewLine}{inputScript}{Environment.NewLine}======");
      using(var utf8Writer = new StreamWriter(process.StandardInput.BaseStream, new UTF8Encoding(false))) {
         utf8Writer.Write(inputScript);
      }
         
      process.WaitForExit();


The most important bit is:

Code: Select all

      using(var utf8Writer = new StreamWriter(process.StandardInput.BaseStream, new UTF8Encoding(false))) {
         utf8Writer.Write(inputScript);
      }


This is where I am forcing the stdin to be fed UTF-8. Without it nothing works, and it took me hours to figure out what I was doing wrong.

[paulmansour]

Thanks Stefano.

By the way, have you looked at the stderr after running your code? I have been using the same technique, more or less, to launch a Dyalog session from Dyalog (with a workspace to load), and all the stuff that gets written to the session goes to stderr. However, any unicode chars get messed up, and no amount of manipulation with ⎕UCS sets them right.

It appears to be a bug, and it is on my to-do list to get a repro to Dyalog. Just wondered if you had same problem...

[StefanoLanzavecchia]

Yes, I confirm what you are seeing. The funny thing is that I tried running the interpreter in terminals hosted by Visual Studio Code and depending on the computer (different OS, different environments, different I don't even know what), in one the terminal would be corrupted and in the other it wouldn't. A few months later, I cannot get it NOT to corrupt the terminal session any more, no matter where I run it.
So, yes: it would be nice if somebody managed to get to the bottom of it or recommend a terminal where this does not happen. I haven't tried Microsoft's latest, for instance: https://github.com/microsoft/terminal It can now be installed directly from the Microsoft Store despite being a very early preview. I simply haven't had time to test it but it might (and it's probably worth trying anyway since it's bound to become the new host for the command prompt, if I understand correctly Microsoft's strategy).

[paulmansour]

Ok, thanks.

Using your example, I got things running from Dyalog, with no files. I NEVER would have figured out in a million years to do:

      sw←⎕NEW StreamWriter p.StandardInput.BaseStream

so thanks again!

Note that I did not need to add the

      ⎕NEW UTF8Encoding 0

to the constructor as I think it defaults to that.

It also took me a bit to realize that I don't need to (and should not) convert to UTF-8 what I feed to the StreamWriter - Dyalog must do this for me I guess.

[paulmansour]

OK, here is my function for executing APL from APL:

Code: Select all

ExecuteAPL←{
     ⍝ ⍺ ←→ Command Line Arguments e.g.: '-b maxws=1g default_io=1'
     ⍝ ⍵ ←→ One or more expressions to execute
     ⍝ ← ←→ Exit Code, Session Output
     ⍺←''
     in←∊(⊆⍵),¨⎕UCS 13
     ⎕USING←'System'∘,¨'' '.IO' '.Diagnostics,System.dll'
     p←⎕NEW Process
     i←p.StartInfo
     i.FileName←2 ⎕NQ'.' 'GetCommandLine'
     i.Arguments←⍺
     i.RedirectStandardOutput←1
     i.RedirectStandardInput←1
     i.RedirectStandardError←1
     i.UseShellExecute←0
     _←p.Start ⍬
     sw←⎕NEW StreamWriter p.StandardInput.BaseStream
     op←{6::0 ⋄ ⍺⍺ ⍵}
     _←sw.Write op⊂in
     _←{6::0 ⋄ z←sw.Close}0
     r←⎕NS''
     r.SessionOutput←p.StandardError.ReadToEnd
     _←p.WaitForExit op ⍬
     r.ExitCode←p.ExitCode
     r
 }

This makes creating clean workspaces very easy:

Code: Select all

       r←'-b' ExecuteAPL'a←⍳10' 'b←7' 'c←a+b' '0 ⎕SAVE ''c:\ports\savetest.dws'''  '⎕OFF 99'
       r.ExitCode
99
       r.SessionOutput

       r←'-b' ExecuteAPL')LOAD c:\ports\savetest.dws' 'a b c' '⎕OFF 99'
       r.ExitCode
99
       r.SessionOutput

c:\ports\savetest.dws saved Fri Oct  4 14:09:03 2019                                                             

 0 1 2 3 4 5 6 7 8 9  7  7 8 9 10 11 12 13 14 15 16



Thanks Stefano!

I have sent this off to Dyalog as it easily shows the bug related to APL chars:

Code: Select all

        r←'-b' ExecuteAPL'''⍒⍋'''  '⎕OFF 99'
        r.ExitCode
99
        r.SessionOutput

⍒⍋   

Another question for you or anyone:

Is there any reason to read StdOut? Does Dyalog do anything with StdOut, or is there a way for the APL programmer to send something to StdOut? ⎕ARBOUT maybe?

Reading both StdErr and StdOut complicates things a bit as it must be carefully threaded or things will hang up - though I have the code for this.

[gil]

This is definitely interesting, I like it!

I have an answer to your question about the garbled output. The StandardError stream doesn't always use UTF-8 encoding, which is why Stefano likely saw it behave correctly some times. On my Win10 it uses System.Text.SBCSCodePageEncoding. If you use the same trick as when writing you can force it to be UTF-8:

Code: Select all

 ExecuteAPL←{
     ⍝ ⍺ ←→ Command Line Arguments e.g.: '-b maxws=1g default_io=1'
     ⍝ ⍵ ←→ One or more expressions to execute
     ⍝ ← ←→ Exit Code, Session Output
     ⍺←''
     in←∊(⊆⍵),¨⎕UCS 13
     ⎕USING←'System'∘,¨'' '.IO' '.Diagnostics,System.dll'
     p←⎕NEW Process
     i←p.StartInfo
     i.FileName←⊃2 ⎕NQ'.' 'GetCommandLineArgs'
     i.Arguments←⍺
     i.RedirectStandardOutput←1
     i.RedirectStandardInput←1
     i.RedirectStandardError←1
     i.UseShellExecute←0
     _←p.Start ⍬
     sw←⎕NEW StreamWriter(p.StandardInput.BaseStream Text.Encoding.UTF8)
     op←{6::0 ⋄ ⍺⍺ ⍵}
     _←sw.Write op⊂in
     _←{6::0 ⋄ z←sw.Close}0
     r←⎕NS''
     sr←⎕NEW StreamReader(p.StandardError.BaseStream Text.Encoding.UTF8)
     r.SessionOutput←sr.ReadToEnd
     _←p.WaitForExit op ⍬
     r.ExitCode←p.ExitCode
     r
 }

[paulmansour]

Gil, excellent! Thanks.

So it's not a Dyalog bug.