Templot Club Archive 2007-2020
|
|||
| author | remove search highlighting | ||
|---|---|---|---|
|
posted: 2 May 2020 10:45 from: Andrew Hunt click the date to link to this post click member name to view archived images |
Hi everyone, As alluded to elsewhere on the forum, I've been experimenting with making Templot run natively on Linux. The good news is that I managed to get it running without too much trouble, albeit with some parts disabled (which will of course need fixing for my work to be of any real-world use). Firstly, a disclaimer: I've never really used Templot until this week. During my research of modelling in P4, Templot came up time and time again, so it was obvious I should learn how to use it - but then I noticed it was currently Windows only, and I figured I might as well start by trying to make it run on Linux (of course I skipped the easy way out of trying Wine - or even starting up the Windows installation I have at home). While we're on the disclaimers, I've never even built a layout, beyond a Hornby loop laid out on the floor as a child. In other words: when it comes to this whole modelling business I'm a bit of a novice. So, back to the topic - what needs adjusting for Linux (this is not a complaints list  - it's merely a list of things that I'll need to work on):- There are quite a few calls to ShellExecute for opening folders and websites, I've tackled this by consolidating into open_url and open_folder methods which can then contain the code to support all platforms. There are a few more calls for images that for now I've commented out. - Some API's from the 'Windows' unit aren't available, but most of those can by replace by LCLType/LCLIntf on Linux. - Some API's for Window and Mouse management aren't available at all, those are commented out for now (but I can hopefully find a solution for Linux). - Some code using the htmlviewer didn't compile, I need to investigate this more because there was no obvious reason for the failure. - Image rendering using EMF isn't available when compiling on Linux, but it should be possible to come up with a replacement. Now the more interesting part isn't actually Linux specific: some Templot code seems to assume you're on a 32 bit system, specifically some parts of intarray_get/set, and some of the users of intarray (e.g. the marks list, which puts pointers in the intarray). So even after I'd done the above work, I was getting segfaults when running Templot. Most Linux users are on 64 bit, and there's no good support for 32 bit software on a 64 bit system (it's possible, but not well supported because most of the time you'd just recompile for 64 bit). Hence I'm trying to migrate this code to work on both 32 bit and 64 bit: - I could have just switched intarray to store int64 (I did do this initially), but that seems wasteful for those parts of the code that only need ints. - But then I experimented with dynamic arrays of Tmark pointers, which appears to work just as well. (It looks like dynamic arrays weren't added to Delphi/Pascal until well after Templot was started, but it's hopefully safe to use them now?) So, here's a screenshot TemplotMec running on a 64-bit Linux system. Time to start learning how to use it (while I iron out the missing portions as detailed above) - don't worry I won't be building anything quite like what you see on screen, this was just an experiment and an attempt to understand pegs and notches: 3665_020544_400000000.png  Now at some point I'm going to have to start up my Windows installation after all, just to make sure the changes I'm making don't break Templot on Windows while I make it work on Linux. I suspect the changes I'm making could also be useful for running Templot on Mac - especially the 64-bit related changes, as I believe they've blocked 32-bit programs on Macs nowadays. I won't bother looking into that until I've got everything working on Linux first. |
||
|
posted: 2 May 2020 14:47 from: Martin Wynne
click the date to link to this post click member name to view archived images |
Hi Andrew, Wow. That's amazing to see Templot running natively on something other than Windows/Wine. I do have a Linux/Ubuntu system, but I use it only to test Templot on Wine, I wouldn't know where to begin in developing natively for it. How long have you been working on that behind the scenes? Bearing in mind that you joined Templot Club only this week? If you've been reading back through these topics, you probably realise that my main motivation in open-sourcing Templot a couple of years ago was to ensure that it remains available if I get run over by a bus. There are quite a few users with lifetime projects who would be stuck if Templot vanished with me. The possibility of getting it ported to other platforms wasn't uppermost in my mind, although of course it all helps to keep it alive if a future version of Windows takes objection to it. Not all of it could be open-sourced because it includes some copyright components for which I purchased licences. Since making it open-source the road has been a bit rocky, and the level of interest lower than I expected. Most recently we had a diversion into Gits and Tortoises which left me utterly confused and not knowing which way to turn. I came to the conclusion that my way of working with code was so different from anything that could be called current industry-standard that no-one but me would ever be able to follow it or take it forward. Then you come along, and without mentioning a single Git or Tortoise you have understood it and got it working on a different platform. You probably don't realise how much that means to me. It means that after working on Templot one way or another for 40 years, I now know for sure that it can live on without me. A few things you mentioned: Yes, my first use of Delphi was with Delphi2, which didn't support dynamic arrays. So I kludged my own way round that with intarray_get/set. It's not strictly dynamic, at new_marks_list you can see Templot estimating how much memory space will be needed for each template. Delphi5 introduced dynamic arrays (essentially using the existing string functions), but I always proceeded on the basis of if it ain't broke don't fix it, so we are still using my original code 25 years later. In fact I have never made any use of dynamic arrays, although I have no reason to think they wouldn't work. For anything like that I now always use a TStringList with linked objects. EMF files are the basis of the sketchboard function and the PDF exports, and valuable in background picture shapes, so need a workaround somewhere. The Delphi TMetafileCanvas functions are also not supported in Lazarus, but I did eventually get the EMF stuff working in Templot3 using direct API calls. Updating to 64-bit and other modern stuff: You probably noticed that Templot uses original 8-bit ASCII/ANSI strings, in some places using them as byte arrays. Modern versions of Delphi have switched to 16-bit unicode, breaking a lot of that. Fortunately Lazarus uses UTF-8, so it remains compatible. Presumably your Linux version is doing the same? Early versions of Delphi used the Intel 80-bit extended type as the default float type (maybe they still do?), so I followed suit in Templot. Including in binary files. That caused a major headache for getting Templot going in Lazarus, which treats extended as a 64-bit cross-platform double, breaking Templot's binary files in the process. It's only recently that I got round this by introducing the text and XML-based MECBOX and BGS3 files for transfer between Templot2 and Templot3. Keep up the good work Andrew. If there is anything I can do to help here or on SourceForge let me know. Many thanks for what you are doing. cheers, Martin. |
||
|
posted: 2 May 2020 23:28 from: Rob Manchester
click the date to link to this post click member name to view archived images |
Hello Andrew, I am very impressed too - I have several PC's/laptops etc that are currently running Windows and Templot is one of the few programs that stops me changing all of them over to Linux. Yes, I have tried kludging Templot onto Linux but I found it difficult and frustrating. Thanks for what you have done and please keep us updated on your 'project'.Rob |
||
|
posted: 3 May 2020 15:35 from: Graeme
click the date to link to this post click member name to view archived images |
Great work, Andrew! I am wholly linux-based on my home machines, and a linux version was always a target for me, but I have not even tried to get Templot to run natively on linux yet. I decided that it would be better to first plug the gaps introduced by the switch to Lazarus, so it is great that you have taken the other route. I have been working (albeit more slowly than I would have liked) on the PDF capability which Martin mentioned, and have a (mostly) working version which I will post in a day or two, as I would love to know if it works OK on linux. Similarly, as I said, I am a linux boy, so if you need any help testing (or whatever) I would be glad to lend a hand where I can. Cheers, Graeme |
||
| Last edited on 3 May 2020 15:36 by Graeme |
|||
|
posted: 3 May 2020 23:52 from: Trevor Walling
click the date to link to this post click member name to view archived images |
Hello, As a Linux user can I just say well done. I have used Templot via wine/Linux since I first found and started using Templot. The way Microsoft and Apple are going going gives me the impression if they don't own or control how software works or is used they don't want to know. One only has to look at something like Skype. It worked fine in Linux until Microsoft got hold of it. They changed it to "Skype for Linux" Since then it is more trouble than any sane person could dream up. It is almost as if they are more interested in making "Linux" look like a basket case. I notice the same thing with Windows 10. Any software that wants to run on their system needs to jump through lots of hoops that were not necessary previously. I notice time and again across the internet users of Windows 10 having problems with perfectly good software that hits the buffers when people start using Microsoft's latest operating system. People say Linux is too difficult or some other similar trope but then proceed to use their android device without a second thought. How many people are using obsolete Microsoft operating systems just so they can use a older favourite piece of software? I seem to have got on my high horse so had better dismount. I will post a picture of the view across the fields tomorrow instead. It is so calm an serene. Regards. Trevor. |
||
|
posted: 13 May 2020 18:39 from: Andrew Hunt click the date to link to this post click member name to view archived images |
Hi everyone, All this interest is rather encouraging! I should probably reiterate that I can't guarantee that I'll get anywhere useful myself, but at least this thread can provide some inspiration to others even if I run out of time or get hit by a bus myself. And certainly, as Martin has discovered - open sourcing doesn't guarantee that an army of coders will turn up - but it at least opens you up to potentially having some contributors over time. Martin Wynne wrote: How long have you been working on that behind the scenes? Bearing in mind that you joined Templot Club only this week? I should post another disclaimer: I have actually been following these messageboards for a while, I simply didn't request my account until last week - so my progress wasn't quite as fast as it might seem! I think I first downloaded the sources maybe 2 weeks before my first posting? While on that track: I've been experimenting a bit more with actually using Templot since my last posting, and the most significant thing that I discovered is that I can't actually save files yet (I see an error message, probably because of some of the code I've commented out) - so that's probably the most pressing thing to investigate. Hopefully this should illustrate how far I still have to go - compiling and loading a program on a new operating system is one thing, ensuring full functionality is another. And saving - and opening - files, is pronably one of the most essential things to get working.But for the past week I've been looking into something different: Martin Wynne wrote: EMF files are the basis of the sketchboard function and the PDF exports, and valuable in background picture shapes, so need a workaround somewhere. The Delphi TMetafileCanvas functions are also not supported in Lazarus, but I did eventually get the EMF stuff working in Templot3 using direct API calls. This part will get interesting: EMF support outside of Windows is unfortunately quite poor (and there's no cross-platform Pascal library that I can find). From what I can tell, Microsoft's is the only full implementation. For PDF exports, I'm hoping Graeme's work will help - and I'm looking forward to seeing it! For background picture shapes, meanwhile, the options seem to be: - LibreOffice probably has the strongest non-Microsoft EMF implementation, but that's tighly coupled to LibreOffice. At the very least, people could use LibreOffice to convert EMF to a different format (more on that later), if they need an EMF background - but we probably can't include this directly in Templot. - The aptly named libEMF only seems to support writing EMF, and a limited subset at that. No use for our needs. - There's an incomplete mono gdi+ implementation, that would also be extremely difficult to call from pascal. - Finally, there's libemf2svg, which I experimented with: it's also incomplete, and not widely distributed (my Linux distribution happens to package it, but most don't), but it was interesting to experiment with. So what I did was: - Used libemf2svg to convert emf to svg. - Then used librsvg to render that svg content - librsvg seems to be one of the most widely used SVG libraries. - Both of the above were called from a new and small C library, which I then incorporated into Templot (using h2pas and some tweaks) - that was also an attempt at understanding how feasible it is to use external libraries or C libraries from Pascal. - I then connected that to the background rendering - and it actually works (after a fair amount of debugging trying to understand how to connect everything together). You might note that I still need to do some scaling: 3665_131336_400000000.png  This probably isn't a worthwhile approach for EMF rendering (most Linux systems can't easily access libemf2svg, and compiling it yourself for templot use would be annoying - notwithstanding that libemf2svg only supports 30% of the EMF spec), but SVG support might be very beneficial, as its one of the most common vector image formats out there. Or even if SVG support isn't added, any other vector image format could be added if there's any library out there that can be used via a C API. Martin Wynne wrote: You probably noticed that Templot uses original 8-bit ASCII/ANSI strings, in some places using them as byte arrays. Modern versions of Delphi have switched to 16-bit unicode, breaking a lot of that. Fortunately Lazarus uses UTF-8, so it remains compatible. Presumably your Linux version is doing the same? That's a great question: I have absolutely no idea what's happening interally, but I will try to spend some time figuring it out (I don't have very much Lazarus or Delphi knowledge, so I still have a lot to learn here). I suspect that it is using UTF-8 on Linux too, but I don't actually know for certain. To conclude my overly long posting: as time allows I'm planning keep pootling along with my experimentation. Don't expect too much activity, but I will try to post here if I learn anything useful and/or make progress. However, as I'm just about to order some wagon kits, underframes, and other modelling components - I expect I'll be rather distracted from my coding adventures in the short term future. But at some point I will want to plan a layout and build some track for it, at which point having Templot working on Linux would probably quite useful. |
||
|
posted: 14 May 2020 12:10 from: Graeme
click the date to link to this post click member name to view archived images |
Andrew Hunt wrote: For PDF exports, I'm hoping Graeme's work will help - and I'm looking forward to seeing itHmmmmm - did I say "a day or two"??? I must have meant a week or two!!!  Actually, coming to look at it again and seeing what progress I had made has re-kindled my interest, and I got into fixing some of the little rats and mice problems which caused me to say it was "mostly" working. Of course one thing leads to another and I would now like to get it more than "mostly working" before I let it out into the wild. Part of the reason, I suspect, is that I started working on this quite a while ago, and a number of factors have got in the way, including:
EMF support outside of Windows is unfortunately quite poor (and there's no cross-platform Pascal library that I can find) I looked at a couple of the alternatives, and of course flip-flopped among them over and over. In the end I looked (for the 237th time) at the native Lazarus PDF library (fpPDF) and decided to bite the bullet and use that. Of course, the interface is very different, so that led to many inconsistencies to be overcome, not least of which:
I have written a wrapper around fpPDF to handle all this, and at some point I will back-flow some of it into the PDF unit so both can be simplified. But why bother with all this? You may well ask, but I have gained a number of benefits:
The main item outstanding is your next point, Andrew, the sketchboard. To be honest, I have not really used this feature (other than to produce a few ugly rectangles in the very early days  ), so I am not too sure what functionality we are trying to support here. Some experimentation needed, for sure!Martin, Would it be possible to get a copy of the current sketchboard code? I realise that there will be calls to procedures we don't have because they are in the licensed libraries, but just to see how they are called would be very useful, and obviously the data structures even more so. I have read through the thread between yourself and (primarily) Brian Nichol back in 2010/2011. This has helped a bit, especially seeing the (scarily good) examples there, but if there is a video or any other docs/threads, perhaps showing a typical use of the sketchboard, that would be great too. Cheers, Graeme -------------------------- PS I have absolutely no idea what's happening interally, but I will try to spend some time figuring it out (I don't have very much Lazarus or Delphi knowledge, so I still have a lot to learn here). I suspect that it is using UTF-8 on Linux too, but I don't actually know for certain.Hahaha - that sounds familiar - I was new to model railways, new to prototype track design concepts, new to Lazarus, almost new to pascal (dredging 1,000 year old memories from deep within the grey matter), etc etc. Happy days! PPS However, as I'm just about to order some wagon kits, underframes, and other modelling components ...If you don't mind me asking, where do you plan to get your underframes? |
||
|
posted: 14 May 2020 13:02 from: Martin Wynne
click the date to link to this post click member name to view archived images |
Graeme wrote: I looked at a couple of the alternatives, and of course flip-flopped among them over and over. In the end I looked (for the 237th time) at the native Lazarus PDF library (fpPDF) and decided to bite the bullet and use that.Hi Graeme, Did you ever follow up on: http://github.com/synopse/SynPDF I thought it looked promising, but I haven't looked at it since: topic 3574 I don't think I can give you a copy of the sketchboard code without Nils's permission. I purchased his code, and then extensively modified it to my own ends. His code and mine is so intertwined that I can't see a way to publish one without the other. It's a difficult situation. I will email you. Also the formatted text blocks on the sketchboard use Sergey's TRichView editor (and more EMFs): http://www.trichview.com For T3 I think the only option is to re-write the sketchboard from scratch. Possibly combined with a complete re-write of the background shapes -- which was my original intention. cheers, Martin. |
||
|
posted: 16 May 2020 09:47 from: Graeme
click the date to link to this post click member name to view archived images |
Martin Wynne wrote: Did you ever follow up on:Hi Martin, Yes I did look at that one. To be honest, I can't remember what drove my choice in the end, but I suspect that this one offered TOO much (read "I could not understand half of what it offered") and I did not want to spend a significant amount of time learning how to use it, especially when my ignorance of PDF made it hard to know what it was trying to do. Now, of course, having spent many hours poring over the PDF spec, it does not seem remotely scary. However, I will stick with fpPDF for now., since it will be an easy decision to reverse. This is because I have isolated all the fpPDF-specific stuff into a separate unit (pdf-lib) which accepts calls in the kind of language that pdf-unit speaks and translates them into calls to fpPDF. It is pdf-lib that translates measurements from 600 dpi integers to mm floats, for example, and from Microsoft BGR colours to RGB, etc. etc. Moving to another library instead of fpPDF would simply be a case of creating a new version of pdf-lib, and the code in there is very straight forward.. I don't think I can give you a copy of the sketchboard code without Nils's permission. I purchased his code, and then extensively modified it to my own ends. His code and mine is so intertwined that I can't see a way to publish one without the other. I fully understand and absolutely agree. I did not realise that you would have had his code to change. For some reason I assumed that you would have executable only, and would have needed to write some kind of interface unit, and it was this I was expecting to re-use. As things stand, without a clear and unambiguous statement that we can re-use his code, then I agree we must assume that it is off limits. For T3 I think the only option is to re-write the sketchboard from scratch. Possibly combined with a complete re-write of the background shapes -- which was my original intention. Yes, indeed! Thanks for the clarification, Martin. Cheers, Graeme |
||
| Please read this important note about copyright: Unless stated otherwise, all the files submitted to this web site are copyright and the property of the respective contributor. You are welcome to use them for your own personal non-commercial purposes, and in your messages on this web site. If you want to publish any of this material elsewhere or use it commercially, you must first obtain the owner's permission to do so. |