Search for a command to run...
Download files from git repositories without cloning the entire project
I have a Bachelor's degree in computer science from University of Delhi and I like to work on small open source projects from time to time.
No comments yet. Be the first to comment.
If you have been following along, you might be wondering, "Why are we making this so complicated? Can't we just give one giant AI all the tools and let it figure it out?" You could. And for simple tasks, that works fine. But the moment your project g...
Welcome back. If you are reading this, you probably saw "Model Context Protocol" in the title and thought, "Oh cool, the shiny new thing!" I have a confession to make: The title of this post is a bit of a lie. We are not building a full-blown "MCP Se...
In Part 1 of this series, we drew boxes on a whiteboard and called it "Architecture". Today, we will stop pretending and start building. Most other tutorials on the internet are very basic and teach you how to build a calculator or some other basic s...
Let’s be honest for a second: The world doesn't need another RAG chatbot. If I see one more tutorial on "How to Chat with your PDF," I might scream. Don’t get me wrong, Retrieval Augmented Generation (RAG) is useful. It’s practical. But it is also in...
As a developer with real life experience in building critical systems for Securities and Exchange Board of India (SEBI) that scale and withstand real-world chaos, I’ve firsthand seen how everyone is trying to replace developers with AI tools. But you...
Git is the most popular version control system used by almost all the software projects. However, it is not the only version control software, there are other popular tools like Subversion and Fossil. You can use Subversion on GitHub for managing your source code.
While cloning the entire repository is generally easier, it a a few drawbacks - it takes longer, uses more bandwidth and takes extra space on your hard drive. These issues are exacerbated if you only want a small portion of a humongous repository.
If you want to download files which have been overwritten then you would need to clone the repository and then revert changes, this takes time and requires knowing how to run necessary git commands.
Although you can download single files easily from raw.github.com, downloading multiple files or entire sub-directories is not possible via this method.
There are multiple ways of downloading sub directories, in this article we will explore two methods to achieve this:
There is a excellent tool called download-directory, it can download files and folders from any GitHub repository. You can even download files from different branches and older commits which might have been overwritten. You just need to pass the GitHub URL of the repository and it will be zipped and downloaded for you. it does not work with GitLab
# URL for downloading a particular folder from the head of master branch
https://github.com/trekhleb/javascript-algorithms/tree/master/src/algorithms/sorting
# URL for downloading a folder from an older commit on master branch
https://github.com/trekhleb/javascript-algorithms/tree/7a37a6b86e76ee22bf93ffd9d01d7acfd79d0714/src/algorithms/ml
When working with servers, I often encounter situations where I need to run a tool that requires a desktop environment. We can use the CLI method for such instances. This method is can also be used to make automated shell scrips.
We will use Subversion to achieve this. You can download Apache Subversion from its download page or if you are on Linux, you can easily download it from your package manager.
In the URL, replace tree/master with trunk if you want to clone the latest version from master branch.
# GitHub URL
https://github.com/trekhleb/javascript-algorithms/tree/master/src/algorithms/string
# URL for SVN
https://github.com/trekhleb/javascript-algorithms/trunk/src/algorithms/string
Replace tree with branches if you want to clone a particular banch.
# GitHub URL
https://github.com/TheAlgorithms/Python/tree/Write-for-current-Python/knapsack
# URL for SVN
https://github.com/TheAlgorithms/Python/branches/Write-for-current-Python/knapsack
You can list the files you are about to download by running svn list <url> in your terminal.
svn list https://github.com/trekhleb/javascript-algorithms/trunk/src/algorithms/string
You can download the files using svn export <url> <local_folder>. This creates a directory locally with the content of the specified subdirectory of the project.
svn export https://github.com/trekhleb/javascript-algorithms/trunk/src/algorithms/string algorithms
We can use download-directory and svn command to download sub directories of a git project from GitHub. This is useful if you only want to clone a small portion of a particular repository.
I have not yet been able to successfully download a particular commit via CLI method or download files from other git hosting providers like GitLab. If you know a way to do so, feel free to share your knowledge in the comments.