Questions tagged [numerical-algorithms]
For questions about numerical algorithms used in artificial intelligence. Examples of those algorithms are Q-learning, simulated annealing, or any of the genetic algorithms. You should use this tag if you want to ask something about the numerical nature or analysis of those algorithms and you don't have more specific tags.
7 questions
0
votes
1
answer
137
views
How is catastrophic cancellation dealt with in loss functions?
It just occurred to me that this seems like it should be a very common problem that must have some kind of solution... Yet I'm not sure what it is...
If there is no solution, does this mean once a ...
- 566
1
vote
0
answers
83
views
GANs: Why does iterative gradient descent sometimes optimise $\min_G \max_D V(D,G)$ and sometimes $\max_D \min_G V(D,G)$?
For the following minimax equation for generative adversarial networks (GANs),
$$\min_G \max_D V(D,G) = \mathbb{E}_{\boldsymbol{x}\sim p_{data}(\boldsymbol{x})}[\log D(\boldsymbol{x})] + \mathbb{E}_{\...
- 33
2
votes
3
answers
337
views
Is there any domain in machine learning that solves a problem by using only analytical algorithms?
Most of the algorithms in machine learning I am aware of use datasets and learning happens in an iterative manner given some examples. The examples can also be understood as experience in the case of ...
- 4,172
3
votes
1
answer
1k
views
What is numerical stability?
I came across the phrase "numerical stability" several times. But almost in the same context.
I encountered this word mostly in the analytical formula for batch normalization.
$$y = \dfrac{...
- 4,172
5
votes
1
answer
2k
views
Why is second-order backpropagation useful?
Raul Rojas's book on Neural Networks dedicates section 8.4.3 to explaining how to do second-order backpropagation, that is, computing the Hessian of the error function with respect to two weights at a ...
- 237
1
vote
1
answer
103
views
What are most commons methods to measure improvement rate in a meta-heuristic?
When I run a meta-heuristics, like a Genetic Algorithm or a Simulated Annealing, I want to have a termination criterion that stops the algorithms when there is not any significant fitness improvement.
...
- 445
7
votes
1
answer
8k
views
How is division by zero avoided when implementing back-propagation for a neural network with sigmoid at the output neuron?
I am building a neural network for which I am using the sigmoid function as the activation function for the single output neuron at the end. Since the sigmoid function is known to take any number and ...
- 73