Docker is a popular cross-platform containerization software package. It essentially allows developers to package up their app with all the necessary dependencies and enable it to work on many different platforms without any further configuration.
I use Docker to run a few things on my server, including PostGIS and MySQL database instances, a Wordpress container to host my girlfriend’s food blog, and a container for the Redis key-value store that my interactive web GIS application uses to push data to clients via websockets.
I’ve experimented with Docker containers off and on for the past year or so, mostly using them on my local machine for dev-related stuff. I’ve ran a few Wordpress containers on my current server before, but in the past month or so I started using the PostGIS, MySQL, and Redis containers to with my public facing sites, and that’s where my problems started.
Earlier this week I was browsing Reddit and came across an article detailing how a university lecturer came across a large data leak using Shodan.io, which is a search engine geared towards network security related subjects. I entered the IP address of my server and was quite suprised to see my Redis database details publically available.
$1938 # Server redis_version:3.0.7 redis_git_sha1:00000000 redis_git_dirty:0 redis_build_id:f09a0843cc9876c3 redis_mode:standalone os:Linux 3.13.0-79-generic x86_64 arch_bits:64 multiplexing_api:epoll gcc_version:4.9.2 process_id:1 run_id:bb0448df5f8ce9d578c7dd73f5b6652368da8911 tcp_port:6379 uptime_in_seconds:593246 uptime_in_days:6 hz:10 lru_clock:3690421 config_file: # Clients connected_clients:5 client_longest_output_list:0 client_biggest_input_buf:0 blocked_clients:0 # Memory used_memory:900400 used_memory_human:879.30K used_memory_rss:1634304 used_memory_peak:956480 used_memory_peak_human:934.06K used_memory_lua:36864 mem_fragmentation_ratio:1.82 mem_allocator:jemalloc-3.6.0 # Persistence loading:0 rdb_changes_since_last_save:0 rdb_bgsave_in_progress:0 rdb_last_save_time:1463287753 rdb_last_bgsave_status:ok rdb_last_bgsave_time_sec:0 rdb_current_bgsave_time_sec:-1 aof_enabled:0 aof_rewrite_in_progress:0 aof_rewrite_scheduled:0 aof_last_rewrite_time_sec:-1 aof_current_rewrite_time_sec:-1 aof_last_bgrewrite_status:ok aof_last_write_status:ok # Stats total_connections_received:113 total_commands_processed:87811 instantaneous_ops_per_sec:0 total_net_input_bytes:22290687 total_net_output_bytes:1335177 instantaneous_input_kbps:0.00 instantaneous_output_kbps:0.00 rejected_connections:0 sync_full:0 sync_partial_ok:0 sync_partial_err:0 expired_keys:0 evicted_keys:0 keyspace_hits:1 keyspace_misses:6 pubsub_channels:0 pubsub_patterns:0 latest_fork_usec:3636 migrate_cached_sockets:0 # Replication role:master connected_slaves:0 master_repl_offset:0 repl_backlog_active:0 repl_backlog_size:1048576 repl_backlog_first_byte_offset:0 repl_backlog_histlen:0 # CPU used_cpu_sys:824.02 used_cpu_user:331.38 used_cpu_sys_children:0.04 used_cpu_user_children:0.02 # Cluster cluster_enabled:0 # Keyspace db0:keys=1,expires=0,avg_ttl=0
Full version of report at the time of discovery can be found here
Now, as a bit of a backgrounder, I used UFW to configure my firewall when I first set up my server. I know that it acts as a “front-end” to iptables, but really didn’t know more than that. I assumed that denying all incoming connections by default, and only allowing on certain ports would be a done deal. Apparently not.
My UFW configuration: To Action From -- ------ ---- 80/tcp ALLOW IN Anywhere 22 ALLOW IN Anywhere 443 ALLOW IN Anywhere 80/tcp (v6) ALLOW IN Anywhere (v6) 22 (v6) ALLOW IN Anywhere (v6) 443 (v6) ALLOW IN Anywhere (v6)
It turns out that Docker doesn’t play well with UFW at all! This post explains in detail the measures you need to take in order to have Docker play nice with UFW on Ubuntu. Seeing as I never took those precautions my Docker containers with exposed ports were not being protected by any type of firewall. I logged into the database from my desktop, which was totally not supposed to be possible, to see if anyone left me any presents.
I was presented with two keys left of my server (there were supposed to be zero, as I only use this instance for PUB/SUB capability). Both of them were SSH private keys, one was for a root user for a random machine, the other was for a user at some Russian netsec website that I won’t link here. What were they used for? Who knows. Perhaps they were placed there for botnets to query and inject into vulnerable servers so their C2 nodes could get access, or maybe I was compromised much worse than I realize and those were left as a form of digial graffiti from the attackers. Who knows?
For technologies like Docker, which put complex abstractions over both the file system and network it’s important to understand exactly what each command
or flag you issue does. When I decided to run
docker run --name redis -p 6379:6379 -d redis I had no idea that port 6379 would be exposed, and that UFW
would be useless to protect my machine. At least I was able to figure out what happened, if you do even a cursory glance at Shodan there are thoudands
of machines out there that aren’t as lucky!