Confused about "Changes not staged for commit" after pull and push

Development Learn to Contribute
1

I’ve sync’d my local copy with supercollider/supercollider (upstream)

$ git diff develop upstream/develop

shows no differences

And I’ve also pushed those to my github clone

$ git diff develop origin/develop

Also shows no differences. But

$ git status
On branch develop
Your branch is up to date with 'origin/develop'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   editors/sc-el (new commits)
        modified:   external_libraries/nova-tt (new commits)

I see from my .git/config those are files in submodules… And

$ git submodule
+7b5d95400306232c1faef43620e703f7ae2b49e8 editors/sc-el (7b5d954)
 8ea022ae0057631c7761257369106023048b49a9 editors/scvim (v1.0.0-11-g8ea022a)
 80a10cee3d203c83696a4f7bfe0289f17a58deff external_libraries/hidapi (hidapi-sc-0.8.0)
 c1449edb28ccba86396a46a9989b8795960160a9 external_libraries/link (Link-3.0.2)
 2bdc68bc5704a42578300a4c18411df2405cb307 external_libraries/nova-simd (heads/master)
+103dd25bb9fd0a359fff685d6aefd8028822afcc external_libraries/nova-tt (103dd25)
 a906f6aaddb6c7b9601e38116ee80fc778c9e9c9 external_libraries/portaudio_sc_org (a906f6a)
 728e26e42645d4d70ca65522990f915f47b47a50 external_libraries/yaml-cpp (release-0.5.3-35-g728e26e)

clearly shows those two have changes.

And I had to do

$ git diff --submodule=diff
Submodule editors/sc-el 0e95a1884..7b5d95400 (rewind):
diff --git a/editors/sc-el/CHANGELOG.md b/editors/sc-el/CHANGELOG.md
deleted file mode 100644

....

to actually see the changes to the submodoules.

And it’s actually possible to make git status be more informative by default about submodules

$ git config status.submodulesummary 1
$ git status
On branch develop
Your branch is up to date with 'origin/develop'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   editors/sc-el (new commits)
        modified:   external_libraries/nova-tt (new commits)

Submodules changed but not updated:

* editors/sc-el 0e95a18...7b5d954 (4):
  < Merge pull request #25 from mlang/emacs25

* external_libraries/nova-tt 55da937...103dd25 (1):
  < Merge pull request #4 from dyfer/rw_mutex-win32

no changes added to commit (use "git add" and/or "git commit -a")

So really git config status.submodulesummary 1 should be a suggested default in some SC git setup tutorial…

Actually plain git diff shows the changes to the submodules by default (unlike git status which needed that flag tweak)…

$ git diff
diff --git a/editors/sc-el b/editors/sc-el
index 0e95a1884..7b5d95400 160000
--- a/editors/sc-el
+++ b/editors/sc-el
@@ -1 +1 @@
-Subproject commit 0e95a1884d36297f7684bc8b18d6187976b4cb1f
+Subproject commit 7b5d95400306232c1faef43620e703f7ae2b49e8
diff --git a/external_libraries/nova-tt b/external_libraries/nova-tt
index 55da93741..103dd25bb 160000
--- a/external_libraries/nova-tt
+++ b/external_libraries/nova-tt
@@ -1 +1 @@
-Subproject commit 55da93741e76632579b63004d3378139e32d634f
+Subproject commit 103dd25bb9fd0a359fff685d6aefd8028822afcc

Although specifying branches on the diff command (as in the beginning of this post) still says nothing about submodules (somewhat obviously the status flag only affects the status command), so the diffs between specified repos can still be confusing… especially since

$ git diff --submodule=diff develop upstream/develop
$ git diff --submodule=diff develop origin/develop

also show nothing. So apparently one Must remember not to include any repos in the command to get the submodule status diffed.

Anyhow, how do I do the pull the push business for those submodules? Do I need to fork the submodules on github too?

2

The widsoms of the Internets suggested

$ git submodule update --remote
Submodule path 'editors/sc-el': checked out '0e95a1884d36297f7684bc8b18d6187976b4cb1f'
Submodule path 'editors/scvim': checked out 'd80cf6e84ed4de3fa0cc1560bf3c1de5adad80fe'
remote: Enumerating objects: 83, done.
remote: Counting objects: 100% (83/83), done.
remote: Compressing objects: 100% (21/21), done.
remote: Total 96 (delta 62), reused 80 (delta 60), pack-reused 13
Unpacking objects: 100% (96/96), 11.42 KiB | 34.00 KiB/s, done.
From https://github.com/Ableton/link
   0b77cc2..184a805  master     -> origin/master
Submodule path 'external_libraries/link': checked out '184a80519a71abf0fa689adb44138ced448f3183'
Submodule path 'external_libraries/nova-tt': checked out '55da93741e76632579b63004d3378139e32d634f'
Submodule path 'external_libraries/portaudio_sc_org': checked out '3822b3c30ec68bd514b5adfea2e8147cf52e0649'
Submodule path 'external_libraries/yaml-cpp': checked out 'b659858b19766e91e3e0cc9385419b287ac999cf'

But while that did something, it actually made the “unsync” worse

$ git status
On branch develop
Your branch is up to date with 'origin/develop'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
  (commit or discard the untracked or modified content in submodules)
        modified:   editors/scvim (new commits)
        modified:   external_libraries/link (new commits, modified content)
        modified:   external_libraries/portaudio_sc_org (new commits)
        modified:   external_libraries/yaml-cpp (new commits)

Submodules changed but not updated:

* editors/scvim 8ea022a...d80cf6e (5):
  > Merge pull request #60 from claudiocabral/vim_terminal

* external_libraries/link c1449ed...184a805 (35):
  > ESP32: Use bswap from endian.h

* external_libraries/portaudio_sc_org a906f6a...3822b3c (34):
  < Merge pull request #1 from bagong/sc39pa196

* external_libraries/yaml-cpp 728e26e...b659858 (61):
  > Some small changes as a result of using a static analyzer (#643)

no changes added to commit (use "git add" and/or "git commit -a")

Anyway, instead of fixing some actual bug in SC, I’ve wasted my evening with this… Thanks for all the gits. I wish SC stopped using git submodules. A google search for “git submodules suck”, “git submodules pain in the ass” returns plenty of results!

Apparently the right magic should have been

git submodule update --init --recursive

Thankfully doing the above on top of the broke state seems to have fixed it

$ git submodule update --init
Submodule path 'editors/scvim': checked out '8ea022ae0057631c7761257369106023048b49a9'
Submodule path 'external_libraries/link': checked out 'c1449edb28ccba86396a46a9989b8795960160a9'
Submodule path 'external_libraries/portaudio_sc_org': checked out 'a906f6aaddb6c7b9601e38116ee80fc778c9e9c9'
Submodule path 'external_libraries/yaml-cpp': checked out '728e26e42645d4d70ca65522990f915f47b47a50'
$ git submodule update --init --recursive
$ git status
On branch develop
Your branch is up to date with 'origin/develop'.

nothing to commit, working tree clean

I can confirm from the official docs that that was the right command:

By default, the git pull command recursively fetches submodules changes, as we can see in the output of the first command above. However, it does not update the submodules. This is shown by the output of the git status command, which shows the submodule is “modified”, and has “new commits”. What’s more, the brackets showing the new commits point left (<), indicating that these commits are recorded in MainProject but are not present in the local DbConnector checkout. To finalize the update, you need to run git submodule update :

$ git submodule update --init --recursive

And there’s apparently to automate that too

If you want to automate this process, you can add the --recurse-submodules flag to the git pull command (since Git 2.14). This will make Git run git submodule update right after the pull, putting the submodules in the correct state. Moreover, if you want to make Git always pull with --recurse-submodules , you can set the configuration option submodule.recurse to true (this works for git pull since Git 2.15). This option will make Git use the --recurse-submodules flag for all commands that support it (except clone ).

I’ve lost the plot on the following paras from the docs though

So far, when we’ve run the git submodule update command to fetch changes from the submodule repositories, Git would get the changes and update the files in the subdirectory but will leave the sub-repository in what’s called a “detached HEAD” state. This means that there is no local working branch (like master , for example) tracking changes. With no working branch tracking changes, that means even if you commit changes to the submodule, those changes will quite possibly be lost the next time you run git submodule update . You have to do some extra steps if you want changes in a submodule to be tracked.

In order to set up your submodule to be easier to go in and hack on, you need to do two things. You need to go into each submodule and check out a branch to work on. Then you need to tell Git what to do if you have made changes and then git submodule update --remote pulls in new work from upstream. The options are that you can merge them into your local work, or you can try to rebase your local work on top of the new changes.

But hopefully I can ignore that for now.

My issue still is that I have no idea what’s on my github origin clone, submodule-wise, since I have no idea how to diff submodules with a specific remote (that always seems to output nothing.) But since I’m probably in “detached state” with submodules, it doesn’t truly matter, who knows…

I’ve tried this from the docs

$ git push --recurse-submodules=check
Everything up-to-date

So they submodules are probably synced too and wont cause any diffs on a PR.

(Committing sub-modules “in sync” with main repo still seems to be quite tricky, even in more recent git versions. I can see why boost or Ableton link are separate submodules, but I’m not so convinced the editor plugins and scide need to be too. Anyway, one thing git was definitely good at was creating the new job description of “System Configuration Management Administrator” like that guy on Stack Overflow with ~1M rep points has…)

3

Hi @RFluff. I am one of the maintainers of SuperCollider. I’m sorry this was a frustrating experience for you.

We have a very handy git cheat sheet which has tips for working with submodules and for working with git in general: https://github.com/supercollider/supercollider/wiki/git-cheat-sheet#working-with-submodules. In fact it provides a description of and solution to the problem you are describing here, because it turns out many developers also struggle with understanding and using submodules.

This is linked from our developer reference, here: https://github.com/supercollider/supercollider/wiki#developer-reference. I would encourage you to read those articles if you are still interested in contributing to SuperCollider, they contain a lot of advice and documentation that has been accumulated over the years. Feel free to contribute your own information as well if there is anything you find missing!

Git is a complex tool, and it would take much longer than an evening to learn every aspect of the submodule system, let alone the entire tool. We have tried to document the common workflows and problems new and experienced contributors have encountered.

If you want immediate help with a development problem, it might be a good idea to join our Slack space and leave a message in the #dev channel! People are often quick to respond there.

1 Like