For some unknown reason Ubuntu Tweak refused to install properly so I decided to go a different way and used the command from this blog article.
The command mentioned in the blog is
dpkg -l 'linux-*' | sed '/^ii/!d;/'"$(uname -r | sed "s/\(.*\)-\([^0-9]\+\)/\1/")"'/d;s/^[^ ]* [^ ]* \([^ ]*\).*/\1/;/[0-9]/!d' | xargs sudo apt-get -y purge
If you don't understand what this command does I recommend to dig into the topic. I must admit it is a little bit complex, but the command is working well.
Use it at your own risk!
Now my system is cleaned up and still boots the correct and latest kernel. All other kernel images, headers and modules are completely removed.