Right now I have a private git repository, which I don't want to make public (with custom configurations, f.e. db login credentials).
In general, configuration doesn't belong in your repository.
One common approach is to include a configuration "template" like config.ini.template that is tracked, but require users to copy that to config.ini, which is .gitignored, and load configuration from there.
Another approach popularized by Heroku is to load your configuration from environment variables.
If you choose to go down either of these paths, you should know that simply removing your currently tracked files with git rm --cached will not purge them from your repository. Previous commits will still contain your database credentials.
You could use something like BFG or git filter-branch to purge them from your repository, but this will cause commits to be rewritten which can cause serious problems, especially on shared repositories. I don't recommend this option unless you fully understand and accept the implications.
My preferred approach would be to git rm --cached the files (or possibly git mv config.ini config.ini.template), add them to your .gitignore, and then change the credentials themselves.