OpenTemplot vs Templot5

[Alistair Ward]

Hi all,

I haven't been watching the Forum much recently, so I was somewhat surprised to see the announcement of Templot5 (actually discovered when I did a Templot2 update, and was notified about old files not working with Templot5).

I've been (very) quietly working away on the OpenTemplot project (https://github.com/openTemplot/openTemplot) on my own for the past year or so. Graeme Defty stepped back from the project early last year due to health issues, and I haven't heard anything from him since, despite a couple of attempts to contact him.

There hasn't been as much progress as I would like on OpenTemplot, but I have been taking a very different approach to what I think is happening with Templot5.

My goal is effectively rewriting Templot from the inside out, restructuring the code to what I hope is a more modern, approachable and structured code base (cross-platform as well!) while still retaining the vast amount of domain knowledge that is embedded within Templot.

The major changes I have been working on have been:

• backwards-compatibility with Templot2 (ie always being able to import a .BOX file)
• changing to a text-based, extensible storage format (currently targeting YAML)
• appropriate separation of model and view data
  • models (templates) hold their data in "world-space"
  • views (forms/printer/pdf) provide the required transformations to actually draw models
• full undo/redo capability
• re-organising all structures to reflect data dependencies

Along the way, I have ported the C libyaml library to Free Pascal, and announced its availability in the Lazarus forums (although I don't know if anyone else is actually using it ).

I have also created a framework for handling object persistence and undo/redo capabilities, along with a code generation tool that generates all the required boilerplate code required. This framework is heavily inspired by a framework I used/developed in a previous job (15-20 years ago) when I was developing professionally using Delphi.

This OpenSource release of Templot5 is obviously coming from a much newer Templot base, so clearly has more features (3D printing, I thinking of you!). I've also been working very slowly, so still a long way from having something significantly useful.

With the release of Templot5, I now have to make a decision about the value of continuing on OpenTemplot vs working on Templot5, and whether there is any path towards combining efforts.

Alistair Ward
Christchurch, NZ

 

[Martin Wynne]

Hi all,

I haven't been watching the Forum much recently, so I was somewhat surprised to see the announcement of Templot5 (actually discovered when I did a Templot2 update, and was notified about old files not working with Templot5).

I've been (very) quietly working away on the OpenTemplot project (https://github.com/openTemplot/openTemplot) on my own for the past year or so. Graeme Defty stepped back from the project early last year due to health issues, and I haven't heard anything from him since, despite a couple of attempts to contact him.

There hasn't been as much progress as I would like on OpenTemplot, but I have been taking a very different approach to what I think is happening with Templot5.

My goal is effectively rewriting Templot from the inside out, restructuring the code to what I hope is a more modern, approachable and structured code base (cross-platform as well!) while still retaining the vast amount of domain knowledge that is embedded within Templot.

The major changes I have been working on have been:

• backwards-compatibility with Templot2 (ie always being able to import a .BOX file)
• changing to a text-based, extensible storage format (currently targeting YAML)
• appropriate separation of model and view data
  • models (templates) hold their data in "world-space"
  • views (forms/printer/pdf) provide the required transformations to actually draw models
• full undo/redo capability
• re-organising all structures to reflect data dependencies

Along the way, I have ported the C libyaml library to Free Pascal, and announced its availability in the Lazarus forums (although I don't know if anyone else is actually using it ).

I have also created a framework for handling object persistence and undo/redo capabilities, along with a code generation tool that generates all the required boilerplate code required. This framework is heavily inspired by a framework I used/developed in a previous job (15-20 years ago) when I was developing professionally using Delphi.

This OpenSource release of Templot5 is obviously coming from a much newer Templot base, so clearly has more features (3D printing, I thinking of you!). I've also been working very slowly, so still a long way from having something significantly useful.

With the release of Templot5, I now have to make a decision about the value of continuing on OpenTemplot vs working on Templot5, and whether there is any path towards combining efforts.

Alistair Ward
Christchurch, NZ

@nickom @NoIdea @Rusty @Alistair Ward @graeme @Steve_Cornford @James Walters @Phil G @Paul Boyd @Tim Adams

Hi Alistair,

Good to hear from you. I'm sorry to learn that Graeme hasn't been well. If you make contact please give him my regards.

Your post has come as something of a surprise. Both Adrian (Rusty) and myself tried to contact you via the Issues section on your site: https://github.com/openTemplot/openTemplot , because the project seemed to be abandoned with no activity for many months. Had we been able to do so, events might have taken a different course with regard to the present Templot5 open-source project.

Essentially I came to the conclusion that Templot and 3D plug track was getting too much for me alone to support after all these years, and that shifting to a fully open-source format would allow others to help, and ensure the future of Templot. Using the free Lazarus compiler rather than Delphi, which is now far too expensive for a hobby project. The key to this was finding a way to make the binary BOX files forwards and backwards compatible between Lazarus and Delphi, and in the end I managed to do this quite simply -- after believing for years that it was a major obstacle to progress.

We are now under way with the code open-source on GitHub, and hoping to make a smooth transition for users from Templot2 to Templot5 in the near future: https://github.com/Martin-Wynne/Templot5

Of course this is just the existing code and program structure. A complete re-write as a modern-style software project was way beyond me, and would cause a significant delay for the many users waiting for progress on the 3D plug track project.

Where we go from here is not at all clear -- I confess that I don't fully understand some of the details of what you are trying to do. If there is a way to integrate your work into Templot5 I would welcome that, but only if it is not going to cause a significant delay to getting Templot5 released. I don't think I could continue Templot2 for several more years in parallel with an unfinished Templot5 project -- the object of the exercise was to do a rapid swap to fully open-source development and then pick up where I left off on the 3D plug track.

What is your thinking on this? It would be a great shame for your work on OpenTemplot to go unused.

cheers,

Martin.

 

[Alistair Ward]

Hi Martin,

Firstly, I'm glad to see that you're actively moving to an open source model. Graeme and I were always concerned that Templot2 was going to be a moving target that would be difficult to keep up with. Having a fully working open-source version where all the ongoing development is happening will definitely make things a lot easier.

I'm in complete agreement with the aim of maintaining a working/useful version at all times. Trying to do a complete rewrite from scratch is often referred to as "the Netscape mistake". I'll admit to heading down that path, although I have been trying to work "inside-out".

One of the changes I have been making that could be done in Templot5 is replacing magic numbers with enums. This is something that is reasonably easy to do (the compiler tells you if you get it wrong!), aids program readability a lot, and helps with future extensibility - if you read/write the *text names* of the enums to files instead of the numeric values, then you can change the enum values in the code (inserting values, reordering values) without breaking saved data.

Once Templot5 is established to the point where you're happy that it will be your base for ongoing development, I'm more than happy to go through the changes I have been making with you (and anyone else that is interested) to see if we have a common path to follow. I know the work I have done so far is a pretty radical change in direction, but I hope we can find a way to make gradual, but steady, improvements to the Templot codebase.

I will say that my enthusiasm has been waning due to working alone - I'm all in favour of having a healthy supply of people to bounce ideas and share the load with...

Cheers,
Alistair.

 

[Martin Wynne]

@Alistair Ward

Hi Alistair,

Thanks for your thoughts.

I would have liked to have gone fully open-source in 2018 with the original OpenTemplot release. But there were some obstacles to that:

1. the key problem was the non-compatibility of the binary BOX file format between Lazarus and Delphi. That seemed impossible to fix at the time.

2. some Templot2 functions use licensed 3rd party components which can't be open sourced -- chiefly the native PDF export and the sketchboard engine.

Recently I found a comparatively simple solution to the incompatible files problem (which I could have done 6 years ago if I'd had my wits about me) which was the game changer needed.

For the screenshot historic maps, Windows withdrew Internet Explorer and the TWebBrowser component, so that put the tin hat on that function in Windows 10 and 11, and the lack of an equivalent in Lazarus was no longer an issue. For most users the tiled map option from the NLS web site is better anyway.

For the PDF export, I have created (in Delphi5, not open-source) a helper executable, which creates the PDF file from EMF files exported by Templot5. This isn't ideal, and it's Windows only, but at least it keeps the function available for users of T5 until we can find a suitable open-source replacement. Graeme was working on that, but the key requirement is that it must create PDF page sizes of say 5 metres x 1 metre for roll-paper printing, and most of the "office" PDF solutions don't do that.

For the sketchboard, I have accepted that in the short-term it won't be available in T5. This means that the compatible BOX files are essential, so that T5 users can swap back and forth to T2 if they need the sketchboard. An open-source re-write of the sketchboard engine will be a major task, and is on the back-burner for now.

Not being a professional IT developer, I often find it difficult to follow comments from such. For example I didn't know that I had any "magic numbers" and I'm not too sure what "enums" are?

Many thanks for your offer of help with T5. As soon as we have got it properly under way and users smoothly transitioned to Templot5, we can think about where we go next. I can get my head down on the 3D chairs while others can maybe chart a new path for the user interface and internal working, if that is deemed desirable.

cheers,

Martin.

 

[Alistair Ward]

@Martin Wynne

Hi,

Not being a professional IT developer, I often find it difficult to follow comments from such.

And I often forget about who I'm talking to, and make assumptions...

To explain those terms:

• "Magic numbers" refer any literal values in the code, where the number appears without any apparent explanation/reason etc. In general these should be replaced with either named constants, or "enums".
• "enums" means enumerated types. From Wikipedia:

In computer programming, an enumerated type (also called enumeration, enum, or factor in the R programming language, and a categorical variable in statistics) is a data type consisting of a set of named values called elements, members, enumeral, or enumerators of the type. The enumerator names are usually identifiers that behave as constants in the language.

An example usage could be the switch pattern, which is currently an integer value, with comments everywhere it is used

new_switch_info.sw_pattern:=0; // flag curved switch. 0 = curved planing or straight switch; -1 = semi-curved switch; 1 = double-curved switch.

Instead of having to remember which numeric values mean what, a preferred approach is to declare an enumerated type:

type
TSwitchPattern = (
spStraightOrCurved,
spSemiCurved,
spDoubleCurved
);

The line above then becomes:
new_switch_info.sw_pattern := spStraightOrCurved;

Which doesn't require a comment, as the code is now self-documenting.

While enumerated types can be forced to have particular values, in general we shouldn't care what actual (internal) value is used.

Where things get tricky is when you save/restore enumerated types, as their internal representation is some form of integer. The preferred way is to save/restore them using their text representation - this means that if you come along later and insert a new value into an enumeration, then everything just works

Cheers,
Alistair.


Regards,
Alistair.

 

[Martin Wynne]

An example usage could be the switch pattern, which is currently an integer value, with comments everywhere it is used

new_switch_info.sw_pattern:=0; // flag curved switch. 0 = curved planing or straight switch; -1 = semi-curved switch; 1 = double-curved switch.

Instead of having to remember which numeric values mean what, a preferred approach is to declare an enumerated type:

@Alistair Ward

Hi Alistair,

I confess to being very fond of numbers. I can remember them, and type them easily and quickly without making typos.

You can also do arithmetic on them, which is much more cumbersome with text labels. For example in the above, how would you replace:

if sw_pattern<1 then ...

Also with numbers, you know what a value isn't. For example if I read:

chair_code:=5;

I know that it's not 4, it's not 6, it's not 139.

But if I read

chair_code:=go_to_the_moon;

I don't know whether the alternatives are dont_go_to_the_moon or come_back_from_the_moon or go_to_mars.

Ranges are also much easier with numbers. For example in case statements:

case chair_code of

7..23: begin ...

I can't imagine how you would write that using text labels?

cheers,

Martin.

 

[Alistair Ward]

I confess to being very fond of numbers. I can remember them, and type them easily and quickly without making typos.

I would suggest that you've had 40 something years to learn the numbers

You can also do arithmetic on them, which is much more cumbersome with text labels. For example in the above, how would you replace:

if sw_pattern<1 then ...

Enumerated types have ordering, so you can still write:
if sw_pattern < spDoubleCurved then...
although I find this is not ideal and would prefer:
if sw_pattern <> spDoubleCurved then...

Also with numbers, you know what a value isn't. For example if I read:

chair_code:=5;

I know that it's not 4, it's not 6, it's not 139.

or not 53 or 79 or 3213... The problem with using numbers as tags is that you don't know what the valid values can be without exhaustive searching through code to find all the usages and try to work out what they mean.

But if I read

chair_code:=go_to_the_moon;

I don't know whether the alternatives are dont_go_to_the_moon or come_back_from_the_moon or go_to_mars.

I know that it means "go to the moon". If I need to know what the alternatives are, I go look at the type declaration, and I see in one place what the options are. Even better, if I start typing, the IDE will present me with options as I type.

Ranges are also much easier with numbers. For example in case statements:

case chair_code of

7..23: begin ...

I can't imagine how you would write that using text labels?

As simple as:
case chair_code of
chS1Joint..chXACrossing: begin ...

Note that I'm using a naming convention here that I'm sure you won't approve of

If you look through the Lazarus source code (and most other Pascal source code), the convention for enumerated types is a 2 or 3 character abbreviation (lowercase) for the type, followed by the option (potentially multiple words, PascalCase, no spaces...).

It's fairly well known that programmers spend much more time reading code than actually writing it. This is why writing clear, simple code is really important.

Cheers,
Alistair

 

[Martin Wynne]

I would suggest that you've had 40 something years to learn the numbers

@Alistair Ward

Hi Alistair,

Not really, the 3D chairing has all been done in the last few years.

But if you ask me while I'm walking through a field of cabbages what the code for an L1 chair is, I can tell you it is 5. A P slide chair is 2.

I would be utterly unable to remember the text name, especially if it requires mixed case. I can only do underscores and remain sane.

But my biggest gripe is spaces around operators. I just can't read them in a fixed-width font.

abc := 123; // means a headache within minutes.

abc:=123; // is fine.

cheers,

Martin.