When you enter the city of Solitude for the first time you'll witness the execution Roggvir, a former Solitude Hold guard. The executions stop after that, something we always found to be a bit odd.
Cue Public Executions, a mod that adds daily public executions to the city after the scripted execution event is completed. Oh, and you can capture bandits for execution as well.
The executions start at 4PM during which time the prisoner is lead by the jailer to the chopping block. Between 6PM and 8PM, the executioner (who prepares for the execution starting at 6AM) goes about his business.
You don't have to worry about your favorite NPC getting mixed up in this either. The mod should only use guards and bandits.