Templot5 and plug track - progress discussions

[Martin Wynne]

@Martin Wynne @Steve_Cornford

Hi,

I just created a PR to fix a potentially serious bug that I spotted. I have replaced the marks_list_ptr with a dynamic array of Tmark records.

This removes the Pointer <==> Integer typecasts, that would have been fine with Delphi 5, but will potentially fail miserably with Lazarus on Win64 as Pointers are 64-bit, and integers are only 32-bit.

On a separate note, I would like to continue in this area and remove all the intarray functions, and replace with dynamic arrays, and possibly replace the array of arrays (background marks) with a single array of records.

Alistair.

@Alistair Ward

Hi Alistair,

Thanks for doing that. Does it still need a lot of testing?

I've been aware of the problem for a long time, but the intarray system has been working fine for over 25 years, and at present we are creating a Win32 program, not Win64. I never considered changing until now because Win64 does not support the 80-bit extended type. For this reason the helper executables for PDF creation and file conversion will have to remain as Win32 even if Templot5 goes to Win64 or other platforms. Likewise the sketchboard and TRichView if they also get done as helper executables from Delphi5.

I created the intarray system back in Delphi2 days because Delphi2 did not support dynamic arrays. I could have changed when I upgraded to Delphi5, but intarray was working fine and I always prefer "if it ain't broke don't fix it", when there is always so much else to be done.

Dynamic arrays are not always easy to use because you can't simply assign one to another. You have to use the Copy functions. I think Borland (as it then was) created the dynamic arrays in response to user demand as a sort of kludge on the existing string functions and I never really trusted them. Presumably in Lazarus they have been implemented in their own right and are fully reliable.

Thanks for the PR. I will have a look and see what to do, if you think now is the time to make the change. So far the priority has been simply to get Templot fully working in Lazarus before we start making wholesale changes. I'm reluctant to spend much time making changes which make no detectable functional difference for users, when there is so much else that they are waiting for.

cheers,

Martin.

 

[Martin Wynne]

On a separate note, I would like to continue in this area and remove all the intarray functions, and replace with dynamic arrays, and possibly replace the array of arrays (background marks) with a single array of records.

@Alistair Ward @Steve_Cornford

Hi Alistair,

I have now pressed all the green buttons in sight and pushed/pulled/merged/committed everything.

It all seems to be working fine, so many thanks -- I can see that it was a lot of work.

Please go ahead and replace all the other intarray functions if you wish, including the aq rail lines.

Now that the marks list is dynamic, presumably we don't need to estimate the size of the array in advance:

function new_marks_list(var list: Tmark_array):boolean;

and it can be bumped as new marks are entered at:

procedure fill_mark(p1,p2:TPoint; code:integer; num_str:string);

I will leave you to make all these related changes.

Thanks again,

Martin.

 

[Steve_Cornford]

Hi Martin,
@Martin Wynne @Alistair Ward

I have pushed/pulled bgkeeps_unit.lfm, bgkeeps_unit.pas that includes the latest expanded background details form.
Also colours_unit.lfm, colours_unit.pas that allows the title of the TcolorDialog screen to be set.
Also corrected a bug in print_settings_unit.lfm & print_settings_unit.pas.

But will this have affected any changes that Alistair made?

Steve

 

[Steve_Cornford]

Hi Martin,
before performing a fetch origin my local Lazarus RDE project options / paths looked like this:-

Having perform a fetch origin the current project options/paths screen looks like :-

so when i went to perform a build it notified me that the folder listed in the target file name field did not exist , did I want to create it?

I assume that this is because you are now using your local github folder as your Lazarus RDE.

Steve

 

[Steve_Cornford]

Hi Martin,
The latest bgkeeps_form:-

Steve

 

[Martin Wynne]

The latest bgkeeps_form:-

@Steve_Cornford

Hi Steve,

The backdrop colour (trackpad paper_colour) is not part of a background template. Somehow I have failed to explain things. The bgkeeps dialog is intended to contain the settings for the appearance of the background templates only. The form Caption says "detail on background templates ...".

There will be separate dialogs for the appearance of the control template.

And for the appearance of the trackpad. Which is the one which should contain the screen backdrop colour.

These latter two dialogs haven't been created yet because everything is done in the menus.

(There is already a dialog for the appearance of the background shapes -- bgnd_unit.)

Sorry if I keep labouring the point, but it all seemed fairly simple originally.

cheers,

Martin.

 

[Steve_Cornford]

Hi Martin,

The trackpad colour is called paper_colour -- daft name, it's nothing to do with printing on paper.

Sorry I misunderstood, and took this to mean that I could add a panel for this.

There will be separate dialogs for the appearance of the control template.

And for the appearance of the trackpad. Which is the one which should contain the screen backdrop colour.

These latter two dialogs haven't been created yet because everything is done in the menus.

Therefore I dont think it does any harm to include this panel, because in a way it does affect how the background templates appear.
Once the new dialogs are created it can be removed from here.
I believe the revsied output elements and background display elements dialogs are easier for users to navigate than in and out of the various nested menu options.

(There is already a dialog for the appearance of the background shapes -- bgnd_unit.)

You asked me to include a button to invoke bgnd_unit:-

But added to this background templates dialog could be the group colour, and the highlight colour (templates when clicked).

And possibly some buttons opening bgnd_form (for the background shapes) and brick_form (for the brick colours).

So I added the [ shapes''' ], and then as the bgkeeps_unit relates to background templates, I added the option to access the list of background templates by adding the button [ templates... ], instead of the button [ bricks.. ], as all this really gave you was the brick colour option of a template about to be store, or as a selection colour for export, whereas keep_form lets you access the colours of the actual stored templates.

If any of these features offend please feel free to remove them.

Steve

 

[Martin Wynne]

instead of the button [ bricks.. ], as all this really gave you was the brick colour option of a template about to be store

@Steve_Cornford

Hi Steve,

Yes, but the OnClick event of the brick colour panel sets other things, it is not just about changing the colour. That's why we need a button to access the brick form, rather than setting the colour directly:

procedure Tbrick_form.store_marker_colour_panelClick(Sender: TObject);

But I'm too tired to argue tonight. If you think it would be better to have one big dialog for everything that's fine by me. But it will need the bgkeeps form Caption changed accordingly. Also it would probably need the left-hand half of the current dialog extracted into a separate coloured panel (or a separate dialog as it was originally).

cheers,

Martin.

 

[Martin Wynne]

If you think it would be better to have one big dialog for everything that's fine by me.

@Steve_Cornford

Hi Steve,

Maybe something like this?

cheers,

Martin.

 

[Alistair Ward]

@Martin Wynne @Steve_Cornford

Hi Martin,

..., and at present we are creating a Win32 program, not Win64. I never considered changing until now because Win64 does not support the 80-bit extended type.

This is the reason why I asked about a week ago what version of Lazarus you and Steve were using. Steve is using the same version as I am, which is the 64 bit version. It can create either Win32 or Win64 executables, but the default is 64bit.

I'm curious why Templot needs to use the 80-bit extended type. 64-bit doubles should give ample precision, and they are smaller and faster. Lazarus (the Free Pascal compiler, anyway) automatically changes 'extended' to 'double' on any platform other than Win32 as it is not portable. One of the first changes Graeme Defty and I made in the earlier OpenTemplot was a global search/replace of Extended to Double. Graeme also wrote some conversion routines so we could read older box files and convert 80 bit extended values to 64 bit doubles (in a portable way).

Now that the marks list is dynamic, presumably we don't need to estimate the size of the array in advance:

function new_marks_list(var list: Tmark_array):boolean;

and it can be bumped as new marks are entered at:

procedure fill_mark(p1,p2:TPoint; code:integer; num_str:string);

Even with the array now being dynamic, it is still worthwhile estimating the required size and allocating the array once. Multiple reallocations (to extend the array) are a performance-killer.

I'll check out fill_mark(), and see about making it properly dynamic - leave the initial estimate in place, but have it auto-extend the array if necessary.

Cheers,
Alistair.

 

[Steve_Cornford]

Hi Alistair,
Can you check the units I changed to make sure I have not reverted any of your changes?
Or does Github perform a merge rather than a replace ?
Steve

 

[Alistair Ward]

Hi Alistair,
Can you check the units I changed to make sure I have not reverted any of your changes?
Or does Github perform a merge rather than a replace ?
Steve

GitHub does merges (and tells you if there are any conflicts).

I had a quick look and don't see any problems (and it still compiles ).

Cheers,
Alistair.

 

[Steve_Cornford]

@Steve_Cornford

Hi Steve,

Maybe something like this?

View attachment 11979

cheers,

Martin.

Stop giving me ideas!

perhaps first we could sketch out an overall schema with similar outlines giving us a path to go along?
Steve

 

[Alistair Ward]

Hi,

I've created another PR to make the marks_list properly dynamic (ie it will auto-extend if required in fill_marks() ).

Cheers,
Alistair.

 

[Steve_Cornford]

Hi Alistair ,
If I had an array of n Trecs, I understand that I could retrieve a record referencing it using an integer valued from 0 to n-1.
Is it possible to define an keyed index to retrieve a record from the array, consisting of 1 or more fields of the record?
Eg key-a consists of field1,field2
Retrieve record using key-a?

Steve

 

[Martin Wynne]

Hi Alistair ,
If I had an array of n Trecs, I understand that I could retrieve a record referencing it using an integer valued from 0 to n-1.
Is it possible to define an keyed index to retrieve a record from the array, consisting of 1 or more fields of the record?
Eg key-a consists of field1,field2
Retrieve record using key-a?

Steve

@Steve_Cornford

Hi Steve,

Not sure what you are asking. You first declare the record types, and then a variable of that type. You can then access the fields of the variable using dots. By convention types begin with an upper-case T, and variables never do.

type
  Tshelf_position=record
                    inches_from_end:double;
                    inches_from_front:double;
                  end;

  Tegg=record
         size:integer;
         boiled:boolean;
         shelf:integer;
         shelf_pos:Tshelf_position;
       end;

var
  eggs:array[1..24] of Tegg;

  n:integer;

begin
  for n:=1 to 24 do begin

    if (eggs[n].size>2) and (eggs[n].boiled=False)
        then begin
               boil_egg(eggs[n].shelf, eggs[n].shelf_pos.inches_from_end, eggs[n].shelf_pos.inches_from_front);

               eggs[n].boiled:=True;
             end;

  end; //next egg

end;

cheers,

Martin.

 

[Martin Wynne]

Hi,

I've created another PR to make the marks_list properly dynamic (ie it will auto-extend if required in fill_marks() ).

Cheers,
Alistair.

@Alistair Ward

Thanks Alistair.

I have merged the changes. I made a slight change to the formatting to remove the spaces around the operator:

  if mark_index>markmax   // increase the length...
     then begin
            SetLength(marks_list, Length(marks_list)+500);
            markmax:=High(marks_list);
          end;

which make code unreadable for me, and also make searching for assignments very difficult.

I am so fed up with GitHub. I have deleted my 556B branch because there was simply no way I could get the latest changes from main into it.

From now on I'm going to work solely in main, and keep my own backups here in case it all goes wrong.

Also, why does GitHub insist on broadcasting my private computer folder locations to the whole world? See Steve's post about his output folder. I have repeatedly added OpenTemplot2024.lpi to .gitignore till I'm blue in the face, but it is still listed at https://github.com/Martin-Wynne/Templot5 as having changed a few minutes ago, and is included in the download.

cheers,

Martin.

 

[Steve_Cornford]

HI Martin,
Just a thought.
Isn't OpenTemplot2024.lpi required by anyone wanting to participate in the project, and therefore should be included in Github?

Could you not set Target file name (-o): of your copy to the relative path name shown below?

Then for testing have a Templot556 destop icon pointing to that folder? (changing it to 557... as needed)

Steve

 

[Alistair Ward]

Hi Alistair ,
If I had an array of n Trecs, I understand that I could retrieve a record referencing it using an integer valued from 0 to n-1.
Is it possible to define an keyed index to retrieve a record from the array, consisting of 1 or more fields of the record?
Eg key-a consists of field1,field2
Retrieve record using key-a?

Steve

Hi Steve,

Would I be right in thinking you come from a database background?

If you have a known, fixed key structure you could use a map data structure. This is typically implemented as a hashmap, although a binary tree structure can also used. For these types you can have an arbitrary structure as the key, and another structure as the data. There are a couple of generic containers in Lazarus that could be used for this.

If you want an arbitrary search capability across some data, we're going to have to do some more work There is always the possibility of putting data into a table and using SQL queries, although this is possibly overkill.

Cheers,
Alistair.

 

[Alistair Ward]

@Martin Wynne @Steve_Cornford
Hi Martin,

Ditto what Steve said re the .lpi file. It needs to be included in git, so the solution is as Steve suggests - make the path relative.

Regarding adding .lpi files to .gitignore, and the file persisting in Git, that is easy to explain. Adding files to .gitignore has no effect on existing files already being tracked by Git - it only prevents newly created files from being show in the Git UI when preparing a commit.

If you truly wanted to remove the .lpi file from Git, you would need to explicitly remove the file from Git, as well as updating the .gitignore file.

Cheers,
Alistair.

 

[Steve_Cornford]

Hi Alistair,

Would I be right in thinking you come from a database background?

Guilty as charged, but don't worry as I am not suggesting we travel down the SQL path.

I believe that you recomended that we avoid multi-dimensional arrays, so was just thinking out aloud.

Steve

 

[Martin Wynne]

Could you not set Target file name (-o): of your copy to the relative path name shown below?

@Steve_Cornford @Alistair Ward

Hi Steve, Alistair,

If I do that \TEMPLOT5_OUTPUT\ will contain any .box and .bgs3 and other files created while testing the compiled exe. You mentioned previously that I had inadvertently pushed some of those. The idea of including \TEMPLOT5_OUTPUT\ in the project folder is to create the blank installation which anyone will need if they download the code. But it needs to remain blank.

I'm therefore using a separate output folder elsewhere -- which is why the path to it is showing in the .lpi file.

cheers,

Martin.

 

[Martin Wynne]

I believe that you recomended that we avoid multi-dimensional arrays, so was just thinking out aloud.

??? What's wrong with multi-dimensional arrays?

Martin.

 

[Martin Wynne]

If you truly wanted to remove the .lpi file from Git, you would need to explicitly remove the file from Git

@Alistair Ward

Hi Alistair,

How do I do that? I've hunted high and low for a delete button on GitHub.

cheers,

Martin.

 

[Alistair Ward]

@Martin Wynne

Hi Martin,

I am so fed up with GitHub. I have deleted my 556B branch because there was simply no way I could get the latest changes from main into it.

You can definitely merge stuff between branches, but I don't know what process you are using to be able to say what the problem is.

What I suggest is that we set up an online meeting some time so you can share your screen and I can see what is happening.

All we need to do that is to coordinate around my work hours, time zones and whatever other activities we have between us...

Cheers,
Alistair.

 

[Steve_Cornford]

Hi Alistair,
@Alistair Ward @Martin Wynne

Sorry I might have jumped to a conclusion when I read this:-

On a separate note, I would like to continue in this area and remove all the intarray functions, and replace with dynamic arrays, and possibly replace the array of arrays (background marks) with a single array of records.

STeve

 

[Steve_Cornford]

Hi Martin,
@Martin Wynne @Alistair Ward

If I do that \TEMPLOT5_OUTPUT\ will contain any .box and .bgs3 and other files created while testing the compiled exe.

I think you will find that .box and .bgs3(and some other extension types) are already listed as files to be ignored by Github.

Steve

 

[Martin Wynne]

Guilty as charged, but don't worry as I am not suggesting we travel down the SQL path.

@Steve_Cornford

Hi Steve,

If you have a list or array of data, it's quite easy to write a few lines of code to replicate an SQL query.

In the case of a TStringList, you can use IndexOf or IndexOfObject to find items:

https://docwiki.embarcadero.com/Libraries/Athens/en/System.Classes.TStrings.IndexOf

I think you will find that .box and .bgs3 (and some other extension types) are already listed as files to be ignored by Github.

Yes but will it actually ignore them?

I don't really trust GitHub to be predictable any more.

cheers,

Martin.

 

[Alistair Ward]

Hi Alistair,

How do I do that? I've hunted high and low for a delete button on GitHub.

cheers,

Martin.

There are (at least) 2 ways to do this:

1) In GitHub, select the file you want to delete. Click the menu in the top right and select "Delete file". You then need to "Commit change".

2) The other method is deleting the file locally. Git responds to changes in your local file system, so if you delete the file locally, and look at changes in Git it will recognise the file has been deleted. If you then commit that change, the file will be deleted from Git as well.

Cheers,
Alistair.

 

[Martin Wynne]

1) In GitHub, select the file you want to delete. Click the menu in the top right and select "Delete file". You then need to "Commit change".

Thanks Alistair. Now done. We shall see if it remains done.

Is there a way to delete it from all the previous versions which it is presumably archiving?

cheers,

Martin.

 

[Steve_Cornford]

Yes but will it actually ignore them?

Yes, it has so far.

As far as .lpi which wasn't set to be ignored from start (as any collaborator needs a copy), if you delete it from Github, I am pretty sure Github will want to delete it from any other contributors local github copy when they perform a fetch origin.

So I will keep a copy somewhere else just in case.

Steve

 

[Martin Wynne]

I am pretty sure Github will want to delete it from any other contributors local github copy when they perform a fetch origin.

I thought the idea of gitignore is that you can have files locally which it ignores?

Martin.

 

[Steve_Cornford]

Hi Martin.
I made sure I had a local copy of OpenTemplot2024.lpi.

I have just gone to the online Github repository, selected my fork.
This showed it was 9 commits behind your Main.
I synced my fork.
then in my local Github desktop, I performed a fetch origin, this created a local pull request, which i then invoked.

The result is that it has deleted openTemplot2024.lpi from my local github folder.

But I can restore it from my local copy and as gitignore is set to ignore it, it will not get copied back up to my fork, and hence wont get merged back into your main.

STeve

 

[Martin Wynne]

I thought the idea of gitignore is that you can have files locally which it ignores?

@Steve_Cornford @Alistair Ward

Yes, just tried it and it has deleted it. I'm really so fed up with GitHub that I'm inclined to abandon using it. I've been here 3 times before and always ended up just scrubbing the whole thing.

Martin.

 

[Steve_Cornford]

I thought the idea of gitignore is that you can have files locally which it ignores?

But only the direction local > online.

It has no effect online > local, as far as I know.

Steve

 

[Martin Wynne]

But only the direction local > online.

It has no effect online > local, as far as I know.

Steve

In that case, why is there a .gitignore file at https://github.com/Martin-Wynne/Templot5 if it is going to take no notice of it?

Martin.

 

[Steve_Cornford]

Hi Alistair, Martin,
@Alistair Ward @Martin Wynne

I believe there is no reason why Martin cannot have two .lpi files in your common Lazaraus RDE/Github local folder.

If so would a compromise be to have two .lpi files in that folder.
For example:-

One called Templot5dev.lpi which has the target path pointing to where Martin wants it to be pointed.
This file to have a specific entry in gitignore such that it is ignored & not distributed.

Another file called OpenTemplot2024.lpi that contains the relative target path (therefore not giving any confidnece away) and which is not set to be ignored in gitignore.

Then Martin just uses Templot5dev.lpi(or whatever file name he wants to use) for his development.

Steve

 

[Steve_Cornford]

Hi Martin,

In that case, why is there a .gitignore file at https://github.com/Martin-Wynne/Templot5 if it is going to take no notice of it?

because your local .gitignore file does not contain an entry to ignore itself, so gets copied to the online repository.

I had a scrambled egg this morning as part of my Michael Mosley fasting day diet

Steve

 

[Steve_Cornford]

Hi Martin,
I have started producing a document containing some screen shots from Templot to aid in any possible sc changes.

I think it will help me understand what bits need to go where and what their labels should be.

On the basis that what is on the label is in the tin....

So far its simply a cut & paste & crop job

Is that ok?
Steve

 

[Martin Wynne]

I had a scrambled egg this morning

@Steve_Cornford @Alistair Ward

Porridge and prunes for me. Scrambled eggs and grilled tomatoes for lunch.

After which I have come to a conclusion.

After several previous failed attempts I have tried to get my head round Git this time. But I have concluded that Git and me just don't get along.

I'm not going to take any further part in GitHub or GitHub Desktop. It is just too frustrating and a hindrance to making progress with Templot. It is one thing too many to be thinking about, one muddle too many to keep extracting myself from.

For the future I shall be posting my files here, where I know exactly where they are, what's in them, and that they can't be changed without my knowledge:

https://85a.uk/templot/companion/templot5_open_source.php

which you can then merge into your own files and/or push to GitHub as you wish.

If you choose to go on using GitHub I will download the files from main when notified of any updates. Alternatively you could post your files or code chunks here on Templot Club in the section set aside for the purpose:

https://85a.uk/templot/club/index.php?forums/6

I can then merge any changes into my files here, and post the updated files at:

https://85a.uk/templot/companion/templot5_open_source.php

which I hope to do every few days.

I'm sorry if all this is disappointing, but something had to change if I'm going to progress Templot in good heart.

Martin.