PID Controller Optimization: A Gradient Descent Approach | by Callum Bruce | Aug, 2023


There are several methods available to tune a PID controller. These include the manual tuning method and heuristic methods like the Ziegler-Nichols method. The manual tuning method can be time-consuming and may require multiple iterations to find optimal values while the Ziegler-Nichols method often yields aggressive gains and large overshoot which means it is not suitable for certain applications.

Presented here is a gradient descent approach to PID controller optimization. We will optimize the control system of a car cruise control system subject to a step change in setpoint.

By controlling the pedal position, the controllerā€™s objective is to accelerate the car up to the velocity setpoint with minimum overshoot, settling time and steady-state error.

The car is subject to a driving force proportional to the pedal position. Rolling resistance and aerodynamic drag forces act in the opposite direction to the driving force. Pedal position is controlled by the PID controller and limited to within a range of -50% to 100%. When the pedal position is negative, the car is braking.

It is helpful to have a model of the system when tuning PID controller gains. This is so that we can simulate the system response. For this I have implemented a Car class in Python:

import numpy as np

class Car:
def __init__(self, mass, Crr, Cd, A, Fp):
self.mass = mass # [kg]
self.Crr = Crr # [-]
self.Cd = Cd # [-]
self.A = A # [m^2]
self.Fp = Fp # [N/%]

def get_acceleration(self, pedal, velocity):
# Constants
rho = 1.225 # [kg/m^3]
g = 9.81 # [m/s^2]

# Driving force
driving_force = self.Fp * pedal

# Rolling resistance force
rolling_resistance_force = self.Crr * (self.mass * g)

# Drag force
drag_force = 0.5 * rho * (velocity ** 2) * self.Cd * self.A

acceleration = (driving_force - rolling_resistance_force - drag_force) / self.mass
return acceleration

def simulate(self, nsteps, dt, velocity, setpoint, pid_controller):
pedal_s = np.zeros(nsteps)
velocity_s = np.zeros(nsteps)
time = np.zeros(nsteps)
velocity_s[0] = velocity

for i in range(nsteps - 1):
# Get pedal position [%]
pedal = pid_controller.compute(setpoint, velocity, dt)
pedal = np.clip(pedal, -50, 100)
pedal_s[i] = pedal

# Get acceleration
acceleration = self.get_acceleration(pedal, velocity)

# Get velocity
velocity = velocity_s[i] + acceleration * dt
velocity_s[i+1] = velocity

time[i+1] = time[i] + dt

return pedal_s, velocity_s, time

ThePIDController class is implemented as:

class PIDController:
def __init__(self, Kp, Ki, Kd):
self.Kp = Kp
self.Ki = Ki
self.Kd = Kd
self.error_sum = 0
self.last_error = 0

def compute(self, setpoint, process_variable, dt):
error = setpoint - process_variable

# Proportional term
P = self.Kp * error

# Integral term
self.error_sum += error * dt
I = self.Ki * self.error_sum

# Derivative term
D = self.Kd * (error - self.last_error)
self.last_error = error

# PID output
output = P + I + D

return output

Taking this object-oriented programming approach makes it much easier to set up and run multiple simulations with different PID controller gains as we must do when running the gradient descent algorithm.

TheGradientDescent class is implemented as:

class GradientDescent:
def __init__(self, a, learning_rate, cost_function, a_min=None, a_max=None):
self.a = a
self.learning_rate = learning_rate
self.cost_function = cost_function
self.a_min = a_min
self.a_max = a_max
self.G = np.zeros([len(a), len(a)])
self.points = []
self.result = []

def grad(self, a):
h = 0.0000001
a_h = a + (np.eye(len(a)) * h)
cost_function_at_a = self.cost_function(a)
grad = []
for i in range(0, len(a)):
grad.append((self.cost_function(a_h[i]) - cost_function_at_a) / h)
grad = np.array(grad)
return grad

def update_a(self, learning_rate, grad):
if len(grad) == 1:
grad = grad[0]
self.a -= (learning_rate * grad)
if (self.a_min is not None) or (self.a_min is not None):
self.a = np.clip(self.a, self.a_min, self.a_max)

def update_G(self, grad):
self.G += np.outer(grad,grad.T)

def execute(self, iterations):
for i in range(0, iterations):
self.points.append(list(self.a))
self.result.append(self.cost_function(self.a))
grad = self.grad(self.a)
self.update_a(self.learning_rate, grad)

def execute_adagrad(self, iterations):
for i in range(0, iterations):
self.points.append(list(self.a))
self.result.append(self.cost_function(self.a))
grad = self.grad(self.a)
self.update_G(grad)
learning_rate = self.learning_rate * np.diag(self.G)**(-0.5)
self.update_a(learning_rate, grad)

The algorithm is run for a specified number of iterations by calling execute or execute_adagrad. The execute_adagrad method executes a modified form of gradient descent called AdaGrad (adaptive gradient descent).

AdaGrad has per-parameter learning rates that increase for sparse parameters and decrease for less sparse parameters. The learning rate is updated after each iteration based on a historical sum of the gradients squared.

We will use AdaGrad to optimize the PID controller gains for the car cruise control system. Using AdaGrad, the gradient descent update equation becomes:

AdaGrad gradient descent update step

Now we need to define our cost function. The cost function must take a vector of input parameters as input and return a single number; the cost. The objective of the car cruise control is to accelerate the car up to the velocity setpoint with minimum overshoot, settling time and steady-state error. There are many ways we could define the cost function based on this objective. Here we will define it as the integral of the error magnitude over time:

Car cruise control cost function

Since our cost function is an integral, we can visualize it as the area under the error magnitude curve. We expect to see the area under the curve reduce as we approach the global minimum. Programmatically, the cost function is defined as:

def car_cost_function(a):
# Car parameters
mass = 1000.0 # Mass of the car [kg]
Cd = 0.2 # Drag coefficient []
Crr = 0.02 # Rolling resistance []
A = 2.5 # Frontal area of the car [m^2]
Fp = 30 # Driving force per % pedal position [N/%]

# PID controller parameters
Kp = a[0]
Ki = a[1]
Kd = a[2]

# Simulation parameters
dt = 0.1 # Time step
total_time = 60.0 # Total simulation time
nsteps = int(total_time / dt)
initial_velocity = 0.0 # Initial velocity of the car [m/s]
target_velocity = 20.0 # Target velocity of the car [m/s]

# Define Car and PIDController objects
car = Car(mass, Crr, Cd, A, Fp)
pid_controller = PIDController(Kp, Ki, Kd)

# Run simulation
pedal_s, velocity_s, time = car.simulate(nsteps, dt, initial_velocity, target_velocity, pid_controller)

# Calculate cost
cost = np.trapz(np.absolute(target_velocity - velocity_s), time)
return cost

The cost function includes the simulation parameters. The simulation is run for 60 seconds. During this time we observe the response of the system to a step change in setpoint from 0 m/s to 20 m/s. By integrating error magnitude over time, the cost is calculated for every iteration.

Now, all that is left to do is run the optimization. We will start with initial values of Kp = 5.0, Ki = 1.0 and Kd = 0.0. These values give a steady, oscillating response, with overshoot, that eventually converges to the setpoint. From this start point we will run the gradient descent algorithm for 500 iterations using a base learning rate of š¯›¾=0.1:

a = np.array([5.0, 1.0, 0.0])
gradient_descent = GradientDescent(a, 0.1, car_cost_function, a_min=[0,0,0])
gradient_descent.execute_adagrad(500)



Source link

Leave a Comment