• 10 dec 2017: forum version update. In case of issues use this topic.
  • 30 nov 2017: pilight moved servers. In case of issues use this topic.
Hello There, Guest! Login Register


Thread Rating:
  • 0 Vote(s) - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
GIT related issues
#1
Music 
I have noticed, that some of our users have questions regarding the handling of GIT.
Purpose of this thread is to provide a unique thread for discussing git related topics here and to summarize results in a FAQ document.
 
Reply
#2
For your Git post and hopefully manual I have the following suggestions:

A Best practice guide for pilight

I have cloned pilight to my own repository because I was afraid to mess things up (well, as I did). I’m wondering if this is necessary.
Questions I have are: If I clone pilight, do I clone stable and development or can (should) I clone development only, and how do I do that?

On the RPi I do:

git clone https://github.com/MyGitUserName/pilight.git --branch development

I now have a copy of pilight (development) on my RPi (either from my own pilight repository on github or direct from the pilight/pilight repository).

I start editing files, run ./setup.sh a lot of times and eventually I have a working patch that I want to commit.
I did:

git commit -a

.. but should first have done a:

git add libs/pilight/protocols/433.92/alecto_ws1700.c ???

For every file I have edited or added?
Than I did:

git push

but that should have been a “git push origin development”? and in between I had to use:

git log --all --oneline HEAD -2
and
git log --all -—decorate —-graph --oneline HEAD -2

but .. how do I “read” the output? How can I tell everything is alright?

Next step is:

git push origin development -f

(or just "git push”?)

Than I go to my GitHub webpage and do a pull-request (?)

Now it gets interesting .. I get a reply that I have to change something to get my changes committed to the main development repository (is that the correct frase?)

Than it gets all blurred for me. How do I proceed? What do I have to do to get my second try as one (1) pull-request to the pilight development repository….

You mentioned that I should have created “a branche”. How do I do that?

What does “git checkout development” do?

Can I get “up-to-date” with the pilight development repository (branche?) without removing my local changes? Do I have to do that?

When my changes are committed to pilight/pilight (development) do those changes automatic appear in my github and on my RPi or do I have to do something to let that happen?

Now I want to make an other change (well, what can I say?).
Do I remove everything from my RPi (and on github) and re-clone everything?
 
Reply
#3
The pilight manual provides sufficient information for users to download the the development branch of the projects github repository and manually compile pilight.

Purpose of this post is a 1st attempt to explain the GIT basics required for developers to successfully submit changes for acceptance by the projects maintainer. Readers are encouraged to participate, discuss the subject and improve the documentation.

1. GIT Repositories
(06-18-2016, 08:56 AM)TheWheel Wrote: A Best practice guide for pilight

The pilight project is managing all project source files on the github.com server in a bare repository named pilight.git.
Write access to this bare repository is limited to the maintainers of the project using the pilight account.
Developers have to provide pull requests to the maintainers in order to get their modifications incorporated.
Maintainers will review the code and enforce certain standards for modules submitted, if modifications are required, they will be discussed on github.
Developers are encouraged to open a thread on the forum for modifications/additions they want to add to the pilight project and to include a link to the respective forum thread together with the commit message. This will ensure that modifications are documented and traceable.
In the following I will assume that the server used is github.com.

(06-18-2016, 08:56 AM)TheWheel Wrote: I have cloned pilight to my own repository because I was afraid to mess things up (well, as I did). I’m wondering if this is necessary.
Yes, it is, as
1. you have no write access rights to the pilight.git base repository.
2. you can not use a bare repository for development, as it lacks the staging area and the working tree.

In a working repository you have three areas for development and file handling:
a) working tree
b) staging area
c) local repository

The files of the working tree are used for editing and compilation.
Modified files are then added to the staging area (using git add <filename> )before they are saved in the local repository (git commit) and identified by a unique commit id.
The commit id is a 40byte hash sum generated based on the content of the files, the authors name and the time data was committed to the local repository.

"git checkout <branch-name>" moves all files from the local repository to the staging area and the working tree and the HEAD to the commit id associated with <branch-name>.

Summary: The hash code of a commit_id identifies all files in the filesystem that belong each other.
The local repository is then pushed to the developers github.com account and a pull request is issued.
The maintainers of the project will review the modifications and honour the pull request once they have accepted the modifications.

What is important to know: Each commit has exactly one (and only one) predecessor, but zero, one, or multiple successors.
If there is no successor, this commit is the tip of a development cycle.
If there is one successor, this commits identifies an older release.
If there are more then one successor, the commits identifies a fork in the development cycle.

A branch-name can be assigned to an individual commit id, typically a tip with no successor.
A branch-line can be built based on the predecessors of the commits back to the root commit_id.
 
Reply
#4
2. GIT repositories / working Tree / Staging Area / Local Repository

(06-18-2016, 08:56 AM)TheWheel Wrote: Questions I have are: If I clone pilight, do I clone stable and development or can (should) I clone development only, and how do I do that?

I assume that with stable you mean the master branch and with development the development branch.

With the knowledge of the above you should be able to answer the question yourself.

A branch name identifies a commit_id. Each commit_id always identifies the complete set of files in the development cycle (similar to a snapshot in other version control systems).
You want to do development on the master files, you do need to clone the master branch (stable), etc.
It should be noted that you can not clone by "commit id" and that there is a difference between checking out a commit by branch-name and by commit id.

As a general rule, if space on your storage medium is of no concern, for developers I recommend to clone the complete git repository, if you want to develop/modify the stable branch, you do need to checkout the master branch, but please note that no pull request will be accepted for the master branch.
For users who want to compile the latest development version only, it is sufficient to clone an individual branch.

(06-18-2016, 08:56 AM)TheWheel Wrote: On the RPi I do:
git clone https://github.com/MyGitUserName/pilight.git --branch development

I now have a copy of pilight (development) on my RPi (either from my own pilight repository on github or direct from the pilight/pilight repository).
You should accustom yourself to standard handling of your GIT account.
When you clone directly from pilight, ensure that authors name and e-mail address are properly set. When you finally push your local repository, ensure that the path is set to your own repository (I assume you do not have the password for the pilight directory !)
So in short, you should fork the main pilight repository to your own pilight directory, configure it appropriately and clone your local repository from there.

In GIT you have a working area, a staging area and the repository.

I noticed that you prefer to use "git commit -a". I STRONGLY RECOMMEND NOT TO USE THE -a OPTION

With "git commit" you tell git to take all changed files from the staging area to your repository.
I know it is a challenge to always add the changed files to the staging area.
With "git commit -a" you copy all changed files from your working area to the staging area and to the repository.
As you typically do not want to have all files from your working to be copied back into the repository, you should not use the -a option, as you need to remove from the staging area before you commit your work the files you do not want to commit using the command "git reset <files>

(06-18-2016, 08:56 AM)TheWheel Wrote: I start editing files, run ./setup.sh a lot of times and eventually I have a working patch that I want to commit.
1.
GIT has no patch concept, as you are used to from other version control systems. With each commit GIT is basically taking a snapshot of the complete filesystem, storing the result in its database and calculating a hash code. GIT is recording the predecessor of every commit, in short: Every commit has one predecessor and Zero, One, or Multiple successors. Due to the fact that the hash code is based on the data content, the name of the author and the time the commit took place, data integrity is an inherent built-in function.

2.
Unlike most other version control systems, GIT has no branches. A branch-name in GIT identifies a single commit hash, nothing more, nothing less. Using the branch-name GIT is able to track back all commits, and therefore can automatically detect all changes made. Normally the branch-name points to the tip of series of commits (e.q. a commit with no successor).

3.
When you check out a branch the associated snapshot is taken from the repository and copied to the working area. Same when you checkout a commit, in this case (if you are not checking out a branch) GIT is entering a mode called detached head. You can assign a new branch-name to this commit (and GIT is no longer in detached Head state)
 
Reply
#5
3. How to commit changes and squash them into a single commit

(06-18-2016, 08:56 AM)TheWheel Wrote: I did:
git commit -a
.. but should first have done a:
git add libs/pilight/protocols/433.92/alecto_ws1700.c ???
For every file I have edited or added?

No "git commit -a" adds all changed files to the staging area. In particular with pilight there are certain .h files you do not want to be added, so you should not use "git commit -a" but "git commit" without the -a option.
Without the -a option you have to manually add all edited / changed files to the staging area.

You can easily check which filenames you have changed using the command "git status".

(06-18-2016, 08:56 AM)TheWheel Wrote: Than I did:
git push
but that should have been a “git push origin development”? and in between I had to use:
git log --all --oneline HEAD -2
and
git log --all -—decorate —-graph --oneline HEAD -2
but .. how do I “read” the output? How can I tell everything is alright?

We have discussed that once we have updated our local repository that we need to push the results.
We have not yet discussed that the maintainers of the main pilight repository do not want to take over your complete history log (e.q. they only want to get one single commit id. So we have to get rid of the superfluous commits.

I am bad at counting and prefer to let the computer do this job.
With "git log --all --oneline --decorate --graph HEAD -2" I have checked that 2 is the number of commits i want to squash into a single commit and that i am doing it with the correct commits. "git log" only lists the commit id's but is not doing anything else.

The interesting part is git rebase -i HEAD~2, as this command is calling the editor and in the editor window that opens you will see your two commits in reverse order (e.q. the oldest commit at the top and in the following line the newer commit. You pick now the oldest commit, and with the squash command you are telling git to squash the newer commit into the older one.

Now I do not know what exactly happened in your local repository, but to cut a long story short you created with commit 3939b77 exactly the commit you wanted to.

How do I know this ?
1. It follows immediately the original commit in development.
2. When i do a "git diff 3939b77 22bebd7" no differences are reported, so both commits are the same.
(just try the same with git diff 3939b77 f74c09c and "git diff 22bebd7 f74c09c")

Thus what you wanted to do is remove all newer commits than 3939b77 from the development branch.

How do we achive this goal ? Well by setting the HEAD to 3939b77 in your working tree and in the staging area (the option --soft of the git reset command ensures that moving the HEAD is copied to the working tree and to the staging area.

The next step is now to push the local repository to your github account. As your HEAD is behind its public counterpart you do need to make use of the option -f to force the update.

(06-18-2016, 08:56 AM)TheWheel Wrote: Next step is:
git push origin development -f
(or just "git push”?)

If you use git push, all branches are pushed.
With "git push origin development -f" only the development branch is pushed to your github.com account.
As you can see from the following git log command that your HEAD is behind the tip of your github repository (origin/development)
Code:
root@pi_65:/home/pi/willem_1/pilight# git log --oneline --all --decorate --graph HEAD -5
*   b40e91a (origin/development) Merge branch 'development' of https://github.com/mrWheel/pilight into development
|\
| * 22bebd7 Change protocol Head, ID and Temperature conform specsheet. https://forum.pilight.org/Thread-Fully-Supported-No-brand-temp-humidity-sensor-alecto-ws1700?pid=17682#pid17682
| * f74c09c Corrections in protocol, header/id and temperature https://forum.pilight.org/Thread-Fully-Supported-No-brand-temp-humidity-sensor-alecto-ws1700?pid=17700#pid17700
* | 3939b77 (HEAD, development) Change protocol Head, ID and Temperature conform specsheet. https://forum.pilight.org/Thread-Fully-Supported-No-brand-temp-humidity-sensor-alecto-ws1700
|/
*   a2719cf Merge pull request #285 from wo-rasp/dev_optional

(06-18-2016, 08:56 AM)TheWheel Wrote: Now it gets interesting .. I get a reply that I have to change something to get my changes committed to the main development repository (is that the correct frase?)

Than I go to my GitHub webpage and do a pull-request (?)

Than it gets all blurred for me. How do I proceed? What do I have to do to get my second try as one (1) pull-request to the pilight development repository….

With pushing your work to your github.com repository your job is done.
The pull-request is already issued and remains valid
 
Reply
#6
4. origin / upstream repositories

(06-18-2016, 08:56 AM)TheWheel Wrote: You mentioned that I should have created “a branche”. How do I do that?

1st you checkout the branch development: "git checkout development"

When you want to create a new branch based on development you use the following command: "git checkout -b xyz"
Do not use xyz but a meaning ful name for the branch.

(06-18-2016, 08:56 AM)TheWheel Wrote: What does “git checkout development” do?

It copies all files from your repository to the working tree and to the staging area.

(06-18-2016, 08:56 AM)TheWheel Wrote: Can I get “up-to-date” with the pilight development repository (branche?) without removing my local changes? Do I have to do that?
Not necessarily, as others may provide changes as well.
You can compare the differences between commits using the "git diff sha1 sha2" command. Replace sha1 and sha2 with the respectiv commit id's. This may not work if you do not copy complete repositories but single commits/branches only.

1. Check which branches are defined in your repository:
Code:
root@pi_65:/home/pi/willem_1/pilight# git branch -a
* development
  master
  remotes/origin/HEAD -> origin/master
  remotes/origin/development
  remotes/origin/ir
  remotes/origin/master
  remotes/origin/rewrite
  remotes/origin/s_mfy

Normally this main pilight repository is referred to as the "upstream" repository. If no upstream repository is defined you will get an error message.

Code:
root@pi_65:/home/pi/willem_1/pilight# git fetch upstream deve
fatal: 'upstream' does not appear to be a git repository
fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

root@pi_65:/home/pi/willem_1/pilight# In the above listing no                                                                 "upstream" directories are listed.

As the command failed, we do need to define the URL for the upstream repository:

You do this with the "git remote add" command.
"git remote add upstream https://github.com/pilight/pilight.git"

Now you can update your local repository.

a) "git checkout development"
From your local repository you checkout the branch you want to update.
b) "git fetch upstream development"

Code:
root@pi_65:/home/pi/willem_1/pilight# git fetch upstream development
Von https://github.com/pilight/pilight
* branch            development -> FETCH_HEAD
* [neuer Branch]    development -> upstream/development
root@pi_65:/home/pi/willem_1/pilight#

You update/add the upstream changes to your local repository
c) "git merge upstream/development"

Code:
root@pi_65:/home/pi/willem_1/pilight# git merge upstream/development
Already up-to-date.
root@pi_65:/home/pi/willem_1/pilight#

Use the "git branch -a" command again will show you that the upstream repository is added:

Code:
root@pi_65:/home/pi/willem_1/pilight# git branch -a
* development
  master
  remotes/origin/HEAD -> origin/master
  remotes/origin/development
  remotes/origin/ir
  remotes/origin/master
  remotes/origin/rewrite
  remotes/origin/s_mfy
  remotes/upstream/development

With the commadn "git remote -v" you get the complete URL:

Code:
root@pi_65:/home/pi/willem_1/pilight# git remote -v
origin  https://github.com/mrwheel/pilight.git (fetch)
origin  https://github.com/mrwheel/pilight.git (push)
upstream        https://github.com/pilight/pilight.git (fetch)
upstream        https://github.com/pilight/pilight.git (push)

(06-18-2016, 08:56 AM)TheWheel Wrote: When my changes are committed to pilight/pilight (development) do those changes automatic appear in my github and on my RPi or do I have to do something to let that happen?

As discussed before,
1st - you can not commit them to pilight/pilight.git.
2nd - they are not commited but merged into the ustream pilight directory on github.

You can only commit them to your own github repository.
when you have configured your name and email address correctly, you will get an email notification about the progress of your pull request.

(06-18-2016, 08:56 AM)TheWheel Wrote: Now I want to make an other change (well, what can I say?).
Do I remove everything from my RPi (and on github) and re-clone everything?

You can do so, but in case you have changed files on your Pi you may loose your changes.

This is also why you should not develop new code using the development branch, but you should always create a new branch, based on those branches you want to see the changes to be added.
So if you want to develop a change for the master branch, you checkout the master branch first and then create a new sub_branch,
if you want to develop for the development branch you checkout the development branch first and create a new sub_branch

So the steps in your case are:
1. You checkout the development branch: "git checkout development"

2. Based on the development branch you can checkout another new branch, for example: "git checkout -b my_next_work"
 
Reply
#7
wo_rasp,

Can you make this a sticky threat?
 
Reply
#8
wo_rasp Wrote:As the command failed, we do need to define the URL for the upstream repository:

You do this with the "git remote add" command.
"git remote add upstream https://github.com/pilight/pilight.git"

Now you can update your local repository.

a) "git checkout development"
From your local repository you checkout the branch you want to update.
b) "git fetch upstream development"

Code:
Code:
root@pi_65:/home/pi/willem_1/pilight# git fetch upstream development
Von https://github.com/pilight/pilight
* branch            development -> FETCH_HEAD
* [neuer Branch]    development -> upstream/development
root@pi_65:/home/pi/willem_1/pilight#


You update/add the upstream changes to your local repository
c) "git merge upstream/development"


Does the command "git remote add upstream https://github.com/pilight/pilight.git" takes in account that I have my own clone of pilight in github and that I'm developing against that clone? I still have not found an answer whether or not that approach is useful (gives me some feeling of not to easily screwing things up)..


I did the above and by the time I execute:
Code:
~/pilight# git remote add upstream https://github.com/pilight/pilight.git

~/pilight# git checkout development
Already on 'development'

~/pilight# git fetch upstream development
remote: Counting objects: 1, done.
remote: Total 1 (delta 0), reused 0 (delta 0), pack-reused 1
Unpacking objects: 100% (1/1), done.
From https://github.com/pilight/pilight
 * branch            development -> FETCH_HEAD

~/pilight# git branch -a
* development
  remotes/origin/HEAD -> origin/master
  remotes/origin/development
  remotes/origin/ir
  remotes/origin/master
  remotes/origin/rewrite
  remotes/origin/s_mfy

~/pilight# git merge upstream/development
fatal: upstream/development - not something we can merge

~/pilight# 
Get the feeling the git add step messed with my setup.
 
Reply
#9
The git remote add upstream [etc.] command adds the official pilight repository on GitHub as a remote repository called "upstream". When you cloned your fork of the pilight GitHub repository, it was added as a remote called "origin". So, the answer is yes and no Tongue Yes, because it doesn't make a difference, No, because you specify where to push your changes when running git push; remember this:

git push origin HEAD(or the name of the branch you want to push to)

The "origin" in that command states that you are pushing to your own repository. Note that only few people have push access to the pilight/pilight repository, so you can't do something like "git push upstream HEAD" - so you can't accidentally mess up something for somebody else Wink

From "upstream", you can fetch and merge changes that curlymo makes to the official pilight codebase. To "origin" you can push your own changes and branches - this is your own pilight repository on GitHub.

I'm actually not sure why the merge you are doing isn't working. I use essentially the same procedure...

Code:
git checkout development
git fetch upstream
git merge upstream/development

What does "git branch" output and what does "git log" (on development) output?
 
Reply
#10
(06-30-2016, 12:00 PM)pilino1234 Wrote: What does "git branch" output and what does "git log" (on development) output?

Thanks for your prompt reply.

Code:
~/pilight# git branch
* GPSprotocol
  development

~/pilight# git log -2
commit 08e140e23ce3bcfe953e2cd15fa7501d9e3ee646
Author: TheWheel <Willem@Aandewiel.nl>
Date:   Wed Jun 15 08:52:48 2016 +0200

    Change protocol Head, ID and Temperature conform specsheet.
    https://forum.pilight.org/Thread-Fully-Supported-No-brand-temp-humidity-sensor-alecto-ws1700
    
    Corrections in protocol, header/id and temperature
    https://forum.pilight.org/Thread-Fully-Supported-No-brand-temp-humidity-sensor-alecto-ws1700?pid=17700#pid177
    
    Change protocol Head, ID and Temperature conform specsheet.
    https://forum.pilight.org/Thread-Fully-Supported-No-brand-temp-humidity-sensor-alecto-ws1700?pid=17682#pid176

commit a2719cf492230146edf276baaf056121f3b54533
Merge: 62e3f56 627c98f
Author: CurlyMoo <CurlyMoo@users.noreply.github.com>
Date:   Mon Jun 13 07:00:36 2016 +0200

    Merge pull request #285 from wo-rasp/dev_optional
    
    Update json elements with conftype DEVICES_OPTIONAL

~/pilight# 

The protocol changes (commit 08e140e23ce3bcfe953e2cd15fa7501d9e3ee646) are already committed by curlymo, so I'm al little surprised it stil shows as something that looks like it still needs to be done. Hm..

I created a branch with the name 'GPSprotocol' that I'm now trying to work on to implement that protocol (see my post GPSensor)
 
Reply
  


Possibly Related Threads...
Thread Author Replies Views Last Post
  Config issues from development to stable 8.0 terrar 5 1,508 10-28-2017, 09:56 AM
Last Post: curlymo
  Some issues going from dev to nightly. psbest 4 4,087 01-02-2015, 10:19 PM
Last Post: psbest
  [Solved] Webgui issues in webkit browsers lvdp 7 5,058 02-19-2014, 09:51 PM
Last Post: curlymo

Forum Jump:


Browsing: 2 Guest(s)