⎕USING and Dfns

General APL language issues

Re: ⎕USING and Dfns

Postby paulmansour on Wed Aug 05, 2020 5:29 pm

Stefano, thanks very much for the insights.
paulmansour
 
Posts: 418
Joined: Fri Oct 03, 2008 4:14 pm

Re: ⎕USING and Dfns

Postby Morten|Dyalog on Thu Aug 06, 2020 7:39 am

<moan>
I've said this in public a few times before, but feel I need to say it again: Remember that by using dfns in this way, you are stepping outside the envelope that the inventor imagined when this functional style of programming was designed and implemented. While we will do our utmost to keep things sane, nobody at Dyalog has thought carefully about bending dfns to do procedural and object oriented programming, and you may find that the wheels come off at speed.

Also note that, as we continue to work on implementing compiler technology, it is very unlikely that an APL compiler will support the various tricks that are available to blend lexical and dynamic scope.
</moan>
User avatar
Morten|Dyalog
 
Posts: 451
Joined: Tue Sep 09, 2008 3:52 pm

Re: ⎕USING and Dfns

Postby paulmansour on Thu Aug 06, 2020 6:49 pm

Morten,

Given what Stefano reported, it hardly seems appropriate to blame your users once again. Furthermore, the fundamental issue appears unrelated to trad vs dfn. If I were using a trad function the the problem of having my function run successfully by accident remains. The solution appears to localize the .NET namespace, which in turn creates more problems with threads. Maybe you should focus on fixing what is an increasingly buggy product instead of blaming your users.

Paul
paulmansour
 
Posts: 418
Joined: Fri Oct 03, 2008 4:14 pm

Re: ⎕USING and Dfns

Postby paulmansour on Thu Aug 06, 2020 7:16 pm

Oh, and would that be the co-dfns project I have been funding? The last time I tried compiling a basic cash flow generation program it fell over for a dozen reasons. I can't even say all I got was this lousy co-dfns t-shirt for the effort. I have no intention of trying to compile anything at this point.

Also, perhaps it would be a good idea for Dyalog to write some documentation on what should be used inside a dfn. For example, should the family of component and native file system functions, which are not free of side-effects, be discouraged? How about threads? How about unnamed namespaces? How much of the dfns workspace is itself an abuse of dfns?
paulmansour
 
Posts: 418
Joined: Fri Oct 03, 2008 4:14 pm

Re: ⎕USING and Dfns

Postby Morten|Dyalog on Fri Aug 07, 2020 12:00 pm

paulmansour wrote:it hardly seems appropriate to blame your users once again.


You are right, I was very imprecise in my little moan, which is a typical failing of moans - I apologise for that. To be clear, I agree that the underlying .NET issue, which affects both dfns and tradfns, is a problem that we should try to fix ASAP.

I was reacting to the sharing of the "trick" of assigning a name in a dfn and then expunging it, in order to create a free but localised name, in order for potential VALUE ERRORs to trigger a search as directed by ⎕USING: I believe this goes very much against the grain of the philosophy of functional programming and what I really wanted to suggest was that this kind of code be written as tradfns rather than "strange dfns".

The trick does work in current Dyalog APL and can probably be relied on to continue to work in the interpreter. However, if a compiler has to take into account the use of mechanisms which dynamically manipulate names (like ⎕EX), it becomes increasingly difficult to avoid all the runtime checks that currently limit the speed of the interpreter and (amongst other things) make it hard for code to actually run in parallel.

With respect to the wider question of whether a compiler can support I/O, the general answer is that it will be possible to get the compiler to support file system functions, but will be "stopped in its tracks" every time it encounters such a function. Code running on a GPU will need to stop and call back to the CPU (perhaps even an embedded interpreter) to implement such features. High speed and/or parallel execution are made easier when there are large functional sections that can safely run in parallel.

Regarding the co-dfns project: Aaron Hsu and Dyalog are very grateful for the contributions that were made during his PhD project. Now that Aaron has joined our team full time, we will not be passing the hat around any longer, but will fund continued work on the compiler ourselves. I agree that we haven't see the results that we were looking for yet, hopefully Aaron will have more time to concentrate on making the compiler commercially useful soon. I'll follow up soon and hear whether you have any code samples to contribute, that could help prioritise this work!
User avatar
Morten|Dyalog
 
Posts: 451
Joined: Tue Sep 09, 2008 3:52 pm

Re: ⎕USING and Dfns

Postby Phil Last on Fri Aug 07, 2020 10:53 pm

I agree that it's a mistake to use the "trick" of using ⎕EX to localise an expected implicit result when it's totally unnecessary using an anonymous, local namespace or just reassignment, especially when JS often warned against using quirks of dfns that could as easily be described as oversights and that might one day be "corrected".

But I think it's also a mistake to expect dfn users to have "the philosophy of functional programming" at the forefront of their minds when most of them are probably using them for the more prosaic reason that they use a much simpler paradigm than tradfns.

KEI tried a number of ways to produce a simpler, perhaps more APL-like way to define functions that probably inspired many to undertake similar experiments. So when JS sent me a draft of his "Dynamic Functions in Dyalog APL"* paper in 1996 I sent this operator that I was already using by return.
      .
∇ r←{a}(f dd)w
r←⎕FX 1 5⍴'a←a a'
r←⍎('aw',f)[('⍺⍵',f)⍳f]
⍝ {a} optional left arg
⍝ f string expression in ⍺ & ⍵
⍝ w right arg
⍝ r result of expression
As far as I'm concerned it encapsulates the most useful aspects of dfns: implicit argument(s) and ease of ambivalence.

Yes. JS became an FP aficionado but he wasn't a puritan. Nor, I believe, did he have programming GPUs as a goal when creating dfns.

* later renamed to "Direct ...".
User avatar
Phil Last
 
Posts: 624
Joined: Thu Jun 18, 2009 6:29 pm
Location: Wessex

Re: ⎕USING and Dfns

Postby Morten|Dyalog on Tue Aug 11, 2020 12:04 pm

paulmansour wrote:Furthermore, the fundamental issue appears unrelated to trad vs dfn.

In fact, the bad news is that upon closer investigation, the issue is not actually directly linked to ⎕USING and .NET; it is a more general issue with all system variables in a multi-threaded environment: values can leak in from the surrounding environment. It is amazing this bug has survived for so long; most people clearly have the same value for most system variables in most namespaces, most of the time. The occasional reports of incorrect values have been classified as un-reproducible "cosmic ray events".

The good news is that we now fully understand the issue, so we are a big step closer to solving it. The challenge is to fix it without significant performance penalties in terms of time or space. We'll keep you updated on our progress!
User avatar
Morten|Dyalog
 
Posts: 451
Joined: Tue Sep 09, 2008 3:52 pm

Re: ⎕USING and Dfns

Postby JoHo on Tue Aug 11, 2020 6:45 pm

Would that issue also explain the "cosmic ray"-VALUE ERRORs of well defined variables,
which I did observe rarely in the tradion of fellow countryman Viktor Hess?

As a sidenote from a bystander:
I would vote for more stability, disregarding performance in a first step.

Another thought:
Maybe fork the software in a classic DyalogAPL/W (without threads, etc.),
in analogy to the "single-bladed pocket knive" Ian Sharp mentioned once,
and a fully featured APL with all the bell and whistles, prioritising on stability.
User avatar
JoHo
 
Posts: 37
Joined: Sat Nov 28, 2009 12:51 pm
Location: Austria, EU

Re: ⎕USING and Dfns

Postby Morten|Dyalog on Wed Aug 12, 2020 8:07 am

JoHo wrote:Would that issue also explain the "cosmic ray"-VALUE ERRORs of well defined variables

No, the bug only affects system variables.

JoHo wrote:I would vote for more stability, disregarding performance in a first step.

Unfortunately, it is my impression that most of the people who want multi-threading care about performance.

JoHo wrote:Maybe fork the software...

Hmm. So now we have two separate APL interpreters to implement, document, test and maintain... And most users will continue to use the old one for a decade or three:-). Could be a lack of imagination on my part, but I'm not convinced that quality will be improved.
User avatar
Morten|Dyalog
 
Posts: 451
Joined: Tue Sep 09, 2008 3:52 pm

Re: ⎕USING and Dfns

Postby Morten|Dyalog on Fri Aug 14, 2020 7:17 am

A quick update: we've been mulling over various solutions during this week, and think there is a good chance we can fix this soon. I'm not quite ready to say I think it might be next week, but we're working on it.
User avatar
Morten|Dyalog
 
Posts: 451
Joined: Tue Sep 09, 2008 3:52 pm

PreviousNext

Return to Language

Who is online

Users browsing this forum: No registered users and 1 guest