put legacy application under VCS?

SALT, SPICE, Subversion, etc...

put legacy application under VCS?

Postby askom on Tue Jan 21, 2014 11:25 pm

I am absolutely happy with SALT and git/Github or bzr/Dropbox. Everything
goes just fine when I use it for my application designed with version control
in mind and today's usual practice to put different things into different namespaces.

But now I am in the process of putting very old application (not my!) under version control.
It's large enough (about 2000 functions) and uncountable number of global variables:-(
Everything is sitting in the root.

It is
+/{↑⍴⎕cr ⍵}¨↓⎕nl 3
63441

lines of code...

First, I tried the normal way of dividing the whole ws to namespaces, but
it doesn't work, because of cross-references of functions from different namespaces,
constant using of global variable in a # namespace from anywhere. It's almost
impossible to find references to these globals and change them to root.
Like you have globals sasha, sa, ha and if you you search/replace you'd get right #.sa and #.ha,
but #.sas#.ha, except of #.sasha

I finally decided to go the dirty way:
1. I created several namespaces and ]save them.
2. Functions where not moved, but copied to these namespaces
3. Start an empty workspace
4. ]load all script from point 1.
5. Bring all function to the root. Something like
(⊂'#.wgui.'){⎕fx ⎕cr ⍺,⍵}¨(↓#.wgui.⎕nl 3)~¨' '
for a namespace (script file) wgui
6. Run application, meet a bug and fix it.
7. It doesn't affect namespaces:-(

If I may put corrected #.foo inside wgui, like wgui.foo,
then I'm happy. wgui updates wgui.dyalog in a directory
under git control and everything is fine.

But I can't replace wgui.foo with #.foo. Or it is because of me?

The only way, which is even more dirty, I can see for now is
save my #.foo as a text file and use diff to correct my wgui.dyalog.

Thank you for any suggestions and hints.

/Sasha.
askom
 
Posts: 36
Joined: Fri Apr 09, 2010 7:22 pm

Re: put legacy application under VCS?

Postby DanB|Dyalog on Wed Jan 22, 2014 3:53 am

Good to read that you like SALT.

Reorganizing code can be difficult, specially legacy code.
You'll need to proceed in steps.

Managing global variables:
you can rename them easily.
The user command LOCATE (it used to be called WSLOCATE) can help you with this.
It accepts a number of switches to fine tune the process.
To perform syntactic replacement you can write, e.g.

]locate sa #.sa ha #.ha sasha #.sasha -replace -syntactic
if you know regular expressions you can use them too, e.g.

]locate \b(sasha|sa|ha)\b #.\1 -pattern -replace
of course you can replace #. by whatever namespace you prefer to put your globals in.

If you put them all in a single namespace you can then save this namespace whole.
You can save them individually also.

You can do the same with your utilities, renaming them with the new path (e.g. program MYAPP becomes #.main.MYAPP), or use ⎕PATH (e.g. ⎕LX←'⎕PATH←''...''') - some people dislike ⎕PATH.

Once the namespaces are created and filled you can ]save them using the -convert switch to turn them into scripted form.

If you prefer to save your code in namespaces in files and bring the code back WITHOUT the namespaces you can do

]load \path\to\your\namespaces\n1 -disperse
]load \path\to\your\namespaces\n2 -disperse
]load \path\to\your\namespaces\n3 -disperse
...

or you can ]snap the whole workspace and create a function to bring it back later:

]snap \path\of\your\application -loadfn -convert -makedir -version

this will create a program <load_ws> (on file) that you can modify to include the -disperse switch.
Here I added -convert to convert your namespaces to a script form and -makedir to ensure your folders exist before saving into them.
I suggest you also add -version so you can have multiple versions and can go back if you make a mistake (e.g. ]load x -version=3)

The problem with the -disperse switch is that SALT does not kick in when you modify an object that has been dispersed.
It is better to keep the code in the namespaces so every time you make a modification SALT intervenes and allows you to save the changes.

You can also save each fn/var individually. I prefer my code in a script but this is really a personal choice.
Certainly reading and fixing several files takes longer than reading and fixing a single (albeit larger) one.

Once your code is on file you can link it with a Version Control System.
For example, using SubVersion, you could checkout a (initially empty) repository where you save all your namespaces into (like the \path\of\your\application path above).
When filled, ADD to subversion all .dyalog files that do NOT have a version number (version numbers appears just before the .dyalog extension, like myapp.3.dyalog) and perform a checkin so SubVersion knows of the new material.

Continue work in APL, saving new versions everytime you make a change, and checkin SubVersion every once in a while, typically when you reach a stable position.

I am assuming you know about of a lot of things here. If any is unclear let me know and I will try to clarify.
/Dan
DanB|Dyalog
 

Re: put legacy application under VCS?

Postby askom on Thu Jan 23, 2014 12:42 am

Dan,

Thank you so much for that informative and helpful answer.
Like a short, but deep course in source versioning.

DanB|Dyalog wrote:Good to read that you like SALT.
/Dan


How it is possible to dislike SALT??? It gives the very
new feeling on APL development. I forgot thousands of
questions I asked myself. Like what is the name of those ws,
where it was located and on what machine, is this the
best function I wrote for this task, or I have to look
for the newer version and so on. Now, on any machine
being even far from my office, just pull (bzr) repozitory
to a local folder and ]load the most fresh piece.

DanB|Dyalog wrote:Reorganizing code can be difficult, specially legacy code.
You'll need to proceed in steps.
/Dan


Oh, YES! I need to perform some experiments on what you advised me.
Right now, just a quick questions.

My legacy application is running under Dyalog Classic 12. Are there
many differences on SALT and user commands to v.13? May I do anything
on v.12 or it is better to switch to v.13 for making namespaces/scripts
and so on?

DanB|Dyalog wrote:Managing global variables:
you can rename them easily.
The user command LOCATE (it used to be called WSLOCATE) can help you with this.
/Dan


In my v.13.2 it is still WSLOCATE. Do you use v.14?

DanB|Dyalog wrote: ]locate sa #.sa ha #.ha sasha #.sasha -replace -syntactic
if you know regular expressions you can use them too, e.g.
/Dan


It sound great! But imagine I have to type ]locate this to_that for
a couple of thousands of globals:-( Is there a normal APL function,
which I may use in my cover functions to process the list of ALL
globals at one loop.

DanB|Dyalog wrote: If you put them all in a single namespace you can then save this namespace whole.
You can save them individually also.
/Dan


What do you mean "save them individually"? Text, script, component file, something else?

DanB|Dyalog wrote:Once the namespaces are created and filled you can ]save them using the -convert switch to turn them into scripted form.
/Dan


When I do this I do not use switch -convert. Just ]save namespace path\file and it works well. What is the purpose of "-convert"?

DanB|Dyalog wrote: If you prefer to save your code in namespaces in files and bring the code back WITHOUT the namespaces you can do
]load \path\to\your\namespaces\n1 -disperse
/Dan


WOW! I did this in the very stupid way:-! Why you developed "-disperse", if there is no way to put dispersed functions under SALT control,
or put them _back_ to a farther namespace? What people use "disperse" for?

DanB|Dyalog wrote: or you can ]snap the whole workspace and create a function to bring it back later:
]snap \path\of\your\application -loadfn -convert -makedir -version
this will create a program <load_ws> (on file) that you can modify to include the -disperse switch.
Here I added -convert to convert your namespaces to a script form and -makedir to ensure your folders exist before saving into them.
/Dan


Do not understand "snap" at all. Will ask you later...

DanB|Dyalog wrote: You can also save each fn/var individually.
/Dan


What does it mean? save like WHAT? Using WHAT command? How "individual" functions looks being saved?
Like script/namespace with a single function?

DanB|Dyalog wrote:Once your code is on file you can link it with a Version Control System.
/Dan


I am very far from this point:-( If I get there... I would be happy to ask you more questions.

Well, now I see "end of your citation" command. I will put it at any suitable place manually.
And how do normal people behave? Should I better place familiar ">" before your words?

Oh, forget the very important. Why people do not like []PATH? What wrong may happen?

Also, what do you think on
'foo' []CS '#'
as a first line of function foo?

Thanks again,
Sasha
askom
 
Posts: 36
Joined: Fri Apr 09, 2010 7:22 pm

Re: put legacy application under VCS?

Postby DanB|Dyalog on Thu Jan 23, 2014 8:21 pm

My legacy application is running under Dyalog Classic 12. Are there
many differences on SALT and user commands to v.13?


Yes. User commands have gone through important changes since 13.1. Many new commands have been added and in V14 many have been renamed. Ideally you should be using the latest version of Dyalog but if you can't you should at least be running 13.1 because user command ]UUpdate has been introduced in that version.
]UUpdate allows you to get the latest changes to SALT/User commands without having to upgrade APL itself.

In my v.13.2 it is still WSLOCATE. Do you use v.14?


I use V14 but pretty soon ]LOCATE and all the other new/renamed commands will be available through ]uupdate

Is there a normal APL function,
which I may use in my cover functions to process the list of ALL
globals at one loop.


All user commands can be called under program control. To call command XYZ you can use ⎕SE.UCMD 'XYZ' which will return the result of command XYZ.
In your case you can build a list of all the items you want to rename, e.g. myglobals←⎕NL-2 (edit it if necessary) and do
Code: Select all
   ⎕SE.UCMD 'WSLOCATE ',(⍕{⍵,' #.MyGlobals.',⍵}¨myglobals),' -replace -syntactic'


What do you mean "save them individually"?


You save an individual program or variable (there are restrictions) just like a namespace, e.g.
Code: Select all
   ]save myProgram /path/ -version


What is the purpose of "-convert"?

When you save a non-scripted namespace the program Save (in ⎕SE.SALT) creates a script for it then discards it after filing it.
Because SALT cannot know what has changed in the namespace, any changes you make to individual items in it will be ignored and you will have to manually issue a ]save mynamespace /same/path/ to save any changes you made.
If you wish to retain that scripted form that ]save creates you can use -convert to tell Save to turn your namespace into a script.
Afterwards, when you edit the script, SALT will be able to save the changes automatically.

Why you developed "-disperse"?

Originally ]save only worked on scripts and it was to allow users to put root items in a scripted namespace which was brought back and dispersed in #.
It turned out to be a handy feature where you may want to copy specific items (you can use -disperse=fnA fnB var1 etc) elsewhere.

How "individual" functions looks being saved?
Like script/namespace with a single function?

As I wrote above you can ]save a single program or variable.
In fact, in the new version of SALT you can now get all individual items in a non-scripted namespace saved separately when using ]snap which will build a directory structure mimicking the namespace saved, i.e. namespace N with programs P1 and P2 will generate a folder named N in which files P1.dyalog and P2.dyalog will contain the programs P1 and P2. Furthermore, N.P1 will be "SALTed" (tagged with SALT information) which means that editing it will allow SALT to save changes automatically.

Why people do not like []PATH?

The problem is when reading the code: you can't tell when the code is. Moreover if the same name appears in 2 or more namespaces in ⎕PATH then the order is important. Any failure to set ⎕PATH, or to set it improperly (e.g. wrong order) may result in a different (possibly crash) behaviour.

what do you think on
'foo' []CS '#'

Unless there is a compelling reason to use it I would avoid it. It is very confusing to start with and there is usually a better way to do things.

/Dan
DanB|Dyalog
 

Re: put legacy application under VCS?

Postby askom on Mon Jan 27, 2014 11:24 pm

Dan!

I'm sorry for a delayed thanks.
I dived too deep in the problem:-)

Well, my English is to bad to express
how I appreciate your clean, compact and
very helpful and informative teaching me.

Is there any way to replace function definition
in a script namespace under APL control? I guess, NO.

The process of collecting uncountable amount of beer
for you has been started.

Regards,
Sasha.
askom
 
Posts: 36
Joined: Fri Apr 09, 2010 7:22 pm


Return to Source Code Management

Who is online

Users browsing this forum: No registered users and 1 guest