In my understanding, git as version control system should track all resources or files that are necessary to build a software version.
If a private key file is needed for a server to function correctly (say to communicate with client that has the public key we distribute), then, should this private key file be tracked in git or somewhere else?
No it should not.
A key has different specifications to source code or program.
The product should not work without a key
It is specification of product that it does not work without a key. If you include the key in the distribution you effectively destroy its use.
A key should be easily replaceable.
In fact, sometimes it's needed to regularly replace it. And sometimes a deploy of a new version is a difficult and long process. Combine that with the need to rebuild your project to get a new key, and you have the recipe for a perpetual headache. If you do not require a rebuild to get the new key, then there is no reason to put the key into git - it's not used in building, bundling or testing your app.
Consider code audits.
An auditor needs to have access to source code to verify logic (and sometimes to a working program). They are not supposed to see (and hence be able to memorize) your private keys. I'm not going to go through whether it's realistic to memorize a 2048-bit key by glancing at it - sometimes reviews are done remotely.
A key does not need a history.
A key is not tied to the version of the software in most cases. Key change dates and reasons may be interesting, but those do not need to be available to everyone.
Keys are issued per company or per server, not per application.
Should you ever decide that you need a second server in another country with different laws and a different key, or allow a different company to run your server software as well, this is most important. Those should have different keys. Bundling your key with the app is a no-go then.
Keys should not be public information
Does your intern really need to have access to your production server key? Really? Does your compliance/security department concur?
There is but one key that you may want to include in Git - a test key for an internal test server or a local server. That will ease initial deployment and allow you to change key formats without making a company wide email asking all devs to change their local keys as well, while not invalidating the actual security and integrity of your production server.
As always, this is a non-obvious question only because there are several valid principles in conflict.
The one is that you should version-control everything that is needed to build the project. This is rather important. The other is that secret information should be passed out on a need-to-know basis to minimize the probability of data breaches. This can be extremely important or less important depending on what the consequences would be.
Since you know the circumstances of your organization better than we do, you must make the judgement call about that trade-off. We can only tell you how important things are in general.
If you put a private key it git, it is no longer private. If you don't mind that happening, then do it. If you want to keep the private key private, don't publish it on a public server.
Instead, check in to git instructions on how someone can generate and use their own private key.
I think it's better not, for security reason. Or everyone can access git can get the private key.
A better way may be to put private key and public key somewhere with internet access. Each client can get public key directly without authorization. And the server will get the private key with authorization. It's just a bit thought, since I'm not the export for the domain.
But if the private key kept in git is only for test but not real product, I think it's fine.
If the key is needed to build or run the software at all, then it is part of source and should be versioned. But I don't think that's common case.
If a key is needed to build or run the software, but the key is needed to designate the build as official release, then the key should live on the build server. In version control should be a method used for generating a valid key for testing or a particular key for use in testing. The key may be itself versioned, but not together with the project, because the team members don't actually need it, only the build server or the release manager do.
