The problems you have with calculating DPS from that formula are because what you call BaseDamage doesn't actually seem to be that.
Assuming that all numbers are positive, BaseDamage / ( BaseDamage + Defense ) will always resolve to a floating point number between 1 (inclusive) and 0 (exclusive). It doesn't matter if you have 10 BaseDamage, 1,000 BaseDamage or 1,000,000 BaseDamage, you will always be in that range. Where in that range depends on how the BaseDamage of the weapon compares to the Defense of the target. That means what actually affects the order of magnitude of the realDamage of an attack is mostly the SkillDmg multiplier.
Looking at these mathematical properties, I would not really call this weapon property BaseDamage but rather DefensePenetration. This might actually be an interesting mechanic to have (or not... it's something you need to testplay), but it does not say much about the power level of a weapon.
So how do we solve this problem?
Well, there is no right solution to this problem. But a change which might do what you want to do (twice as powerful weapon = about twice as much damage) is to add the BaseDamage as another multiplicative factor:
float realDamage = SkillDmg * BaseDamage * BaseDamage / (BaseDamage + Defense)
If you like the defense penetration mechanic you accidentally discovered, this is how it would look with Penetration as a separate weapon stat:
float realDamage = SkillDmg * BaseDamage * Penetration / (Penetration + Defense)
The nice thing about this formula is that it scales quite well:
- Even when the defense is pathetic compared to the attack value, there is never more damage than attack. This gives you an upper limit on how much damage a character can inflict, which makes balancing far easier.
- On the other extreme, no matter how high the defense gets, it can never completely mitigate damage (except through rounding errors), so there is always room for improvement for the defender and there is never a completely pointless attack.
- When attack and defense are roughly the same, there is roughly half as much damage as attack. This is true no matter how large the values are. This is also what you can base your DPS estimation on. Simply assume that the enemy has as much defense as the weapon, which means your DPS formula becomes
AttackFrequency * BaseDamage / 2