Maximum Likelihood Parameter Estimation

What is the exponential distribution?

The exponential distribution is a probability distribution that describes time between events in a Poisson process. There is a strong relationship between the Poisson distribution and the Exponential distribution.

Let’s say that you try to model the number of api calls (arrivals) per second towards an LLM inference server. Arrivals per second has a Poisson distribution with arrival rate 100, which means that 100 api calls are made per second. “The expected mean inter-arrival time is 0.05 seconds, because an api can be expected every 0.05 seconds. The inter-arrival process is modeled by the exponential distribution. The units for the Poisson process are api calls and the units for the exponential are seconds.

Videos that may help you understand these distributions:

Task 1

Simulate the interarrival times using an exponential distribution with the rate parameter 𝜆=100

Task 2

Use the stochastic gradient descent (SGD) algorithm to minimize the negative log-likelihood of the exponential distribution. Output: (a) the estimated parameter after a number of iterations of your choice and (b) plot the value of objective function over the iteration index.