IDS. It is normally set up passively in a network to detect attack's on all of the hosts. Not the system it is running on. But I myself have not that much experience in the whole IDS topic.
IPTables is a firewall built into the Linux kernel that you way want to look into.