Have you checked your server logs lately? Did you see those "odd" requests from arbitrary IPs that appear to perform a single request and "vanish"? Have you ever wondered how many of those are actually random? Do they return ? How often?
No matter which service you expose to the internet (http, ssh, smtp, imap), you are certain to notice protocol-aware requests (e.g. valid HTTP get request) from random IP addresses hitting your public services.
The following blog post focuses around answering these questions and the ways we utilize the Abuser module of Echofish to identify persistent attackers on our services, that would otherwise stay unnoticed.