Hey folks,

This is an update to: https://sh.itjust.works/post/580838

On https://github.com/csm10495/commit-ment, I made it to somewhere around 22 million commits. I can’t imagine this is not a world record for commits to a single branch. In the middle of the night, I got an email from GitHub support saying:

A few minutes later, I got another email like so:

I’ve asked them if the two emails are related (and I guess if the first one is some sort of error since there was no personal info in that repo). I’ve also asked if they can give any information about what triggered the email and if they can give me more info about what it looks look on their side.

I’ve also asked if they can re-enable it so I can give one more commit to say the final results on the readme then (public) archive it.

We’ll see what they say.

Doing a pull is interesting at the moment, it shows:

git pull origin master --no-rebase -vvv
ERROR: Access to this repository has been disabled by GitHub staff due to
excessive resource use. Please contact support via
https://support.github.com/contact to restore access to this repository.
Read about how to decrease the size of your repository:
  https://docs.github.com/articles/what-is-my-disk-quota

fatal: Could not read from remote repository.

Please make sure you have the correct access rights
and the repository exists.

Similar thing happens if you try to clone: git@github.com:csm10495/commit-ment.git

So yeah, I figured this would happen sooner or later. I just hope they can tell me a bit more about what it looks like on their side since managing this repo on my box is a pain, I can’t imagine what it could look like on theirs. I’m also curious how pull requests could merge at such a rate given that just doing a pull on my end could take minutes. So many questions!

This whole project was really just for curiosity on my end, so anything I can learn/find out is much appreciated on all ends.

Anyways, just figured I’d update y’all.

  • pe1ucaEnglish
    14·
    1 year ago

    excessive resource use

    What’s the final size of the git folder?
    What’s the final time and CPU usage to process the different git commands?

    • csm10495@sh.itjust.worksOPEnglish
      19·
      1 year ago

      Last check was over 8GB

      A full (or full master branch) clone was well over 4 hours. A shallow (--depth 1) clone would take like 5 seconds. A pull origin master on a local copy that was less than 2 days old was well over an hour.

      I was using GitHub Actions to get the latest ‘total commit count’. I was actually using https://github.com/actions/cache to cache a full repo clone, pull origin master then update that cached copy after to not have to do a full repo clone each time. At that point run a rev-list command to get the count.

      Doing that process would take a few minutes.

      I also recently started having GitHub Actions make more branches to be auto merged into master. They allow actions to be run every 10 minutes, so I was having it make commits for 10 minutes then it would push temp branches that would be PR’d and merged via the gh cli by my locally running instance.

      Just that 10 minutes of commits would be over 4500 commits per branch. Doing the push from that point would take about 2 minutes.

      • JackbyDev@programming.devEnglish
        1·
        1 year ago

        Did you try any blobless or treeless clones? They’re ways to more easily clone large repositories you don’t need the full history of.

  • Erlingur@programming.devEnglish
    10·
    1 year ago

    That’s hilarious! I’m actually impressed it lasted for 22 million commits, I would have thought the breaking point would be earlier.

  • snowe@programming.devEnglish
    9·
    1 year ago

    wow, that’s pretty intense.i hadn’t heard of your project before. was the intent just to see how many commits github could handle?

  • haruki@programming.devEnglish
    8·
    1 year ago

    Congratulation on breaking Github…

    It’s interesting to see that your personal repo can also affect another repo.

  • csm10495@sh.itjust.worksOPEnglish
    5·
    1 year ago

    Tiny mini update: GitHub support has been really nice and all is fine. They told me that this repo was taking up 5.5TB of resources doing certain operations. Also pointed me to https://github.com/github/git-sizer and https://github.blog/2018-03-05-measuring-the-many-sizes-of-a-git-repository/

    They are happy to have the 5.5TB back… and I’m happy to give them back. I’ll probably upload/archive the code and the final notes in a new repo, and then close this chapter.

    If I ever really want to play with this again, I’d get a GitHub Enterprise license and let it go wild on a local box.

    • pythoneer@programming.devEnglish
      1·
      1 year ago

      I think it’s a interesting case that they didn’t expect somehow, I have a real repo that has around 1.5M commits on GHEC and surprisingly it works rather well so far, although some functionality (that I don’t really care about) is slower than usual.

  • canpolat@programming.devMEnglish
    3·
    1 year ago

    How did it go (or how is it going) with the local Git repository? I suspect it’s still working as expected?