Git notes

[Alistair Ward]

Here are some notes on using git, based on my experience. I'm going to cover a bit of higher-level workflows, the tools that I use and some detailed usage notes.

High level flow
A fairly common approach with git that I recommend is to use feature branches for all development.

The main idea of this is that the main branch is always good, releasable code. Any new feature is developed using a branch, and only when it has been tested on the branch (and is theoretically ready for release) is it merged to main

In a diagram, this might look like this:

This shows work being done simultaneously on 2 features, each branched from main.
feature1 is merged to main when it is ready.
feature2 branch is updated from main (this is the point where any conflicts are resolved), and can then be merged back to main.

Key notes:

• main should always in a "good" state
• any conflicts are resolved on the branch before merging back to main.
• the only commits in the main branch should be merges of completed features.

Tools
I use and recommend:

• Git for Windows - https://gitforwindows.org/
• Git Extensions - https://gitextensions.github.io/
• a good merging tool

In my day job I use Beyond Compare (https://www.scootersoftware.com/), which is a very good compare/merge tool. It's not free, but it's not very expensive either.
For Templot-related work, I am currently using Araxis Merge (https://www.araxis.com/). It is normally horribly expensive, but they offer free licenses for contributors to open source projects.
I have seen some people at my work using kdiff3, and I find it rather ugly, and not the easiest to use.
I have not used WinMerge, so can't pass any judgement on that

I have not used GitHub desktop so I don't know what features it offers, or how it compares.

Forks and/or branching
For any repository in GitHub, only the Owner or Collaborators are allowed to directly write into repository.

Forking allows anybody (other Contributors) to suggest changes (ie create Pull Requests (PR's). A Fork is where anybody can create their own full copy of the repository, make changes, and then create Pull Requests back to the original repository. It is entirely up to the Owner/Collaborators of the original repo whether or not they accept the PR.

The Owner or Collaborators can push changes directly to a repo, either to main or to branches. As I suggested above, I think all new features should be pushed to a branch (within the repo), and then merged to main via a PR. There are branch protection rules that can be applied in GitHub to explicitly prevent people from pushing changes directly to main which should be enabled.

Quick summary: the Owner and Collaborators should create branches for new work, and merge those to main using PR's. Other Contributors should make a fork, and then submit PR's from that fork. Whether other Contributors use branches or not is not so important.


I'll add a follow-up post tomorrow with more detailed notes on how I use Git for Windows/Git Extensions to keep my local repo synced, create branches and push work to GitHub.

Alistair.

 

[Alistair Ward]

Part 2...

git setup details

I'm going to describe the process of cloning a git repo, making and committing changes, and creating a Pull Request (PR). This will be from the point of view of a Contributor (ie a person who does *not* have rights to update the main repo directly).

The first step as a Contributor is to create a GitHub account (github.com), find the main repo (https://github.com/Martin-Wynne/Templot5) and create a fork. This is a full copy of the repo in your own personal account on GitHub.

Cloning the Repo

Before creating a local clone of this repo on your PC, you need to make a decision about how to authenticate with GitHub. There are 2 protocols supported: https or git/ssh.

To use https you need to create a Personal Access Token (PAT) which is effectively a special password with limited privileges for accessing github repos. Once a PAT is created it is best managed using the Git Credential Manager (this is how I do things in my day job).

To use git/ssh protocol, you need to create a private/public key pair to use for authentication. You add the public key to GitHub account, and your local git tools use the private key when connecting to GitHub. Instructions for setting up SSH are at https://docs.github.com/en/authenti...h/adding-a-new-ssh-key-to-your-github-account. SSH is my personal preferred option.


Go to your forked repo in GitHub, and copy the URL for the repo.

Click the Code button (1), and then click the Copy button beside the URL (2)

To make your local clone, start a cmd window and go to the directory where you want to put the local repo. Enter the command:

With a little luck this should be the last time you need to use git from the command line

You can now open the OpenTemplot2024.lpi project with Lazarus in the repo folder. Build the project, following the instructions in the README.md file.

Checking git status

Start Git Extensions, and open the Templot5 repo. You'll see a window a bit like this:

The main window shows the commit history for the repo. The Commit button in the toolbar shows number of changed files.

The "main" and "origin/main" show that the local and remote main branches are at the same position. "main" is highlighted to show that it is the current checked-out branch.

Committing and pushing your changes

After making changes in the Templot source code, open the repo in Git Extensions and click the Commit button. A window like this will appear:

The top box on the left shows files that have been changed. You need to stage the files you want to commit by selecting the file and clicking "Stage", or clicking the "Stage all" button if you want to commit all the modified files.

The upper window on the right will show you the diffs for the currently selected file.

Use the lower right window to enter a commit message for your changes and then click the "Commit" button:

After pressing Commit, you return to the main window, and can see the new commit:

You can see now that "main" is one commit ahead of "origin/main", and the "Push" button also shows 1 commit ahead.

You can now push the changes to your GitHub repo by selecting menu Commands > Push, or click the Push button:

A progress dialog will appear, and when complete you return to the main window, and "main" and "origin/main" are now both at the latest commit:

Note that if any changes have occurred on "origin/main" than the Push will fail, and you will need to update your local repo before you can try pushing again.

Once your work has been pushed, you can now create a PR in GitHub, and hope that Martin (or another Collaborator) merges it to the main repo

Next steps are updating your local repo from GitHub...
(to follow in Part 3)

 

[Alistair Ward]

Part 3...

I lied at the end of my previous notes This entry will be a quick look at creating a PR and a brief discussion of why I did things wrong...

The easiest place to create a PR (Pull Request) is in GitHub. Login to GitHub, and go to your forked copy of the Templot5 repo. You will see a status messages telling you how many commits *ahead* of and how many commits *behind* the original repo you are:

From here, you can click the "Contribute" button and then click "Open pull request", which brings you to this:

The top part of the screen gives you important information:
1) that you are requesting to merge from the main branch of your fork to the main branch of Martin's (the original) repo. These settings are editable if required (if you're working with other branches).
2) that you have no conflicts that would prevent a merge. If there *are* conflicts, you should update your fork from Martin's repo and resolve the conflicts in your fork. PR's should always be clean merges.

The title and description for the PR will be taken from your commit messages. If you only have 1 commit in the PR this will generally be fine. If you have many commits, you will almost certainly need to update these to summarize the commits.

Under the description you will see the big green "Create pull request" button. Don't click this just yet...

Below this you will see all the added/changed files in your commit(s), and associated diff's. This is your opportunity to review these for any silly mistakes/omissions *before* creating the PR

When you're happy with your PR, go ahead and click the "Create pull request" button.

You now get to wait for either Martin or a Collaborator to accept the PR.

Now this is where I describe the (potential) mistake I made earlier...

I *should* have created a local branch, rather than working directly on main (not strictly essential as a Contributor), but absolutely essential as a Collaborator). By doing this work on my local main, I have made things more difficult for myself if I want to keep doing work while waiting for the PR to be merged.

The next part will be updating my local repo from Martin's repo, along with some conflict resolution...

 

[Martin Wynne]

@Alistair Ward

Hi Alistair,

Many thanks for your detailed notes.

I confess that much of it is going over my head. That may be because I'm classed as the "Owner" and seeing different web pages.

I'm using GitHub Desktop which seems to provide everything I need. I imagine others here are doing the same. Perhaps you could cover GitHub Desktop in your notes?

In your latest PR to me you say "If you need to add a file in the folder to git you can still do this by manually staging the file from the command line".

I'm sorry, I don't know what "manually staging a file from the command line" means. Perhaps you could explain. Thanks.

You have asked that all files in the \TEMPLOT5_OUTPUT\ folder be ignored. I don't want to do that because the \internal\hlp\ folder contains some images which are used within the templot_5.exe program. Anyone downloading the Zip file and wanting to try compiling Templot5 will need them. Also the other folders structure is needed when running Templot5, although not essential. Those folders are currently empty, and any file types they might contain are already in the .gitignore section.

Thanks,

Martin.

 

[Alistair Ward]

Thank you @Martin Wynne for the feedback, which makes this a nice teaching moment

GitHub PR's are not just a means of merging code, they are also a Code Review mechanism.

Open https://github.com/Martin-Wynne/Templot5 and click on Pull requests at the top:

You will then see a list of the current open PR's:

Click on the PR you are interested in to view the details:

Clicking "Files changed" brings up all the files that will be changed by this PR and, most importantly, the "Review changes" button.

You can now add comments to specific bits of code, by clicking on the little blue "+" that appears when your mouse is over code in the diff windows.

You can add a single comment that is recorded in GitHub immediately, or you can start a review. A review allows you to group a bunch of comments together, and then you can add a final overall review comment by clicking "Review changes". Personally, I hardly ever use the "Add single comment" function and will nearly always start a review.

When you have completed your review, you click the "Review changes" button:

In this case, @Martin Wynne , this is where can put your comments, and then say "Request changes". This tells the author of the PR that their PR won't be merged as-is, and why.

After the PR has been updated, it can be re-reviewed, and then Approved/Merged (hopefully). Note that there are filters on the "Files changed" page - the default is "Changes from all commits". The drop-down allows you to view all changes, changes since your last review, or changes for a specific commit.

I have to get back to my day job, so I'll address the other feedback a bit later...

 

[Alistair Ward]

I don't know what "manually staging a file from the command line" means. Perhaps you could explain.

Back in Part 2 (https://85a.uk/templot/club/index.php?threads/git-notes.994/post-12878) I showed the Git Extensions Commit dialog.

The top-left box shows the modified files that git knows about, and the bottom-left box shows the files that have been "staged". Staged just means that these are files that will be included in the next commit.

"Manually staging a file from the command line" is telling git to stage a file (ie include it in the next commit) using the command-line interface. The reason why you might want to do this is if you some fairly general exclude condition in your .gitignore file, but you have some specific file (that is currently being ignored) that you want to commit anyway.

An example would be if I wanted to commit the templot_5.exe file (which I don't really, having gone to the trouble of removing it...). The command line would look like this:

If I now go back to Git Extensions (because I have a slight allergy to using the git CLI), I can now see that templot_5.exe is staged, ready to be committed:

These is where I carefully Unstage templot_5.exe, as I really don't want to commit it.
I hope this helps explain what is meant by staging.

 

[Alistair Ward]

@Martin Wynne I have amended my PR based on your feedback for .gitignore to only ignore templot_5.exe - if you would like to merge it, I will continue with my git notes...

Alistair.

 

[Martin Wynne]

@Martin Wynne I have amended my PR based on your feedback for .gitignore to only ignore templot_5.exe - if you would like to merge it, I will continue with my git notes...

Alistair.

@Alistair Ward

Hi Alistair,

It's not currently included in the Zip -- I just did a download to check:

I will copy it from my Lazarus folder into my Git folder only when I want to do a release. This is the OpenTemplot option for those folks who want to tinker with the code in Lazarus for their own use without getting involved in Git.

cheers,

Martin.

 

[Steve_Cornford]

Hi Alistair,
In your part 3 you stated:-

Now this is where I describe the (potential) mistake I made earlier...

I *should* have created a local branch, rather than working directly on main (not strictly essential as a Contributor), but absolutely essential as a Collaborator). By doing this work on my local main, I have made things more difficult for myself if I want to keep doing work while waiting for the PR to be merged.

You could edit the previous part & correct your (potential) mistake if you wanted to using the edit option at the bottom of your posts.
Steve

 

[Alistair Ward]

Hi @Martin Wynne , @Steve_Cornford ,

First, templot_5.exe should absolutely *not* be committed to source control. As explained in one of my previous messages, it just does not belong there.

If you want to make templot_5.exe available to people via GitHub, the way to do it is as a *release*. In the GitHub screen capture above, there is an option "Create a new release". I haven't been through this process myself yet, but that is the way to distribute the executable file - either as an Installer, or a zip of the TEMPLOT5_OUTPUT folder.

Regarding going back to fix mistakes, the mistake was to commit work to main, rather than using a branch - not something that can be fixed by editing the forum entry :-(

Alistair.

 

[Martin Wynne]

First, templot_5.exe should absolutely *not* be committed to source control. As explained in one of my previous messages, it just does not belong there.

@Alistair Ward

Hi Alistair,

Ok, I've added it to the .gitignore file.

I'm hoping to release an update to Templot2 in the next few days and include the first release of Templot5 within it. It's needed for the latest updates to the 3D plug track which won't be in Templot2.

cheers,

Martin.

 

[Alistair Ward]

Part 4: where I update my repo from Martin's...

The first step (which is a one-off) is to set up a new remote so I can update my local repo directly from Martin's.

In Git Extensions, select Repository > Remote repositories...

In the dialog I click the "+" button to add a new remote.

I then add the new remote details: I name it "martin", enter the Url, which I copy from GitHub, and save.

I answer "yes" here.

At this point Git does a Fetch from Martin's repo, and I see this in the Git Extensions history window:

And this shows why committing work to main (rather than a branch) was a mistake.
My main and origin/main (my fork on GitHub) are at my last commit, which I submitted as a PR. Martin chose not to accept my PR, but instead made an equivalent change directly (commit 36412a8). For me to bring my main into sync with martin/main, I now have to discard my commits, and I will have to do a force push to sync my fork on GitHub.

If I had made my changes on a branch, then syncing my main would have been very simple. If Martin had accepted my PR, then the merge would have just appeared when I did the fetch. If Martin had not accepted my PR, I could just delete the branch, and no harm done...

As it is, I now need to fix up my main to match Martin's, and then do a force push. A force push is basically a way of forcibly changing history in a repo, and is generally considered to be a very dangerous operation (in my work environment force push is explicitly disabled, due to the havoc it can cause). When a repo is only used by once person you can get away with it, but if you are working with a shared repo, a forced push can seriously screw up a team :-(

My steps are to select martin/main, right-click, and then select Reset current branch to here...

A dialog with reset options appears, and in this case I select "Hard":

After a confirmation dialog, and a progress dialog, I now see this history:

I can now attempt to push to GitHub, using default options:

And I get this error dialog:

At this point, I Cancel, go back to the Push dialog, and select the "Force" option:

I get a dialog asking if I want to use the safer "Force with lease" option, and in this situation it doesn't really matter which is used.
My history in Git Extensions is now:

We can see that main, origin/main and martin/main are now all aligned. My local commits are no longer visible, as there is nothing pointing at them. They will continue to exist in git until some point in the future when git decides to do some housekeeping and delete commits that have no references to them. Until that time, they are still potentially recoverable.

This was not the episode I was intending to cover, but I got a little side-tracked. The next thing I want to cover is resolving conflicts, but that will have to wait a couple of weeks, as I'm gong to be away travelling for work until the beginning of October.

Finally, I'm going to sound like a broken record, and reiterate what I said in another thread...

I think the plan to leave main as-is (the 555A release), and do everything in a new T5-556 branch is a mistake. This will cause confusion for others, as main is where everyone expects that action to be - having to find out the current development branch is going to be a pain.

I'll lay out what I think the workflow should be again:

1. Contributors/Collaborators/Owner(!) develop a feature on a local branch.
2. That local branch is pushed to a fork (Contributors), or to the Martin's repo (Collaborators/Owner).
3. A PR is created and reviewed.
4. The PR is merged to main (in martin/main).
5. Developers pull from martin/main to update their local main
6. ... cycle repeats...

A "feature" is very broadly defined as "a unit of work that is 'releasable' as one piece". This could be anything from a one-line bug fix, to a major new item. It could be a single commit, or many commits, potentially from several developers working together.

One of the key things though, is that all merges to main should happen in Martin's main (GitHub repo), as a result of accepting a PR. All merges from main (to resolve conflicts) should happen locally on a developers PC.

For a release strategy, the recommended approach is as I suggested elsewhere: put a tag on the commit used to create the release, and in the event of needing a "hot-fix", create a new branch from that tag (and don't forget to merge that back into main after releasing the fix!). This means that everyone is always able to get and use main, knowing it will be the latest work.

Alistair.