Operating systems are often exposing severe vulnerabilities. The dirtyCOW vulnerability in Linux Kernel in 2016 – exposed all derived operating systems such as Android, RT-Linux etc., to the same vulnerability which could be exploited for privilege escalation leading to compromise of all security of the device and other devices in the network. Earlier in 2019, one of the most widely used real-time operating system in CPS sector was found to have a severe vulnerability. There were 11 vulnerabilities disclosed which left billions of devices in various critical infrastructures, automotive systems, and avionics systems exposed until they were patched. As a result, one of the major thrusts of research has to be in operating system security – finding vulnerabilities, customizing existing operating systems by restricting APIs for system calls in order to reduce the attack surface, and also to apply formal methods to prove security properties of various system call APIs. Vulnerability of Kernal design (such as DirtyCOW) could have been exposed through formal modelling.
Some of the issues to be tackled at this layer are:
i. Formal Model Driven Fuzz Testing to discover operating system vulnerabilities.
ii. Vulnerability and Penetration Testing Methodology and tools
iii. Security of System Call APIs
iv. Security of Device Drivers
v. Customizing existing O/S with minimal attack surface
vi. Formal proofs of security properties
vii. Trusted Operating System by boot strapping trust from TCB and formal proofs of trust properties
viii. Host Based Intrusion Detection