Docker is a popular cross-platform containerization software package. It essentially allows developers to package up their app with all the necessary dependencies and enable it to work on many different platforms without any further configuration.
I use Docker to run a few things on my server, including PostGIS and MySQL database instances, a Wordpress container to host my girlfriend’s food blog, and a container for the Redis key-value store that my interactive web GIS application uses to push data to clients via websockets.
I’ve experimented with Docker containers off and on for the past year or so, mostly using them on my local machine for dev-related stuff. I’ve ran a few Wordpress containers on my current server before, but in the past month or so I started using the PostGIS, MySQL, and Redis containers to with my public facing sites, and that’s where my problems started.
Earlier this week I was browsing Reddit and came across an article detailing how a university lecturer came across a large data leak using Shodan.io, which is a search engine geared towards network security related subjects. I entered the IP address of my server and was quite suprised to see my Redis database details publically available.
$1938
# Server
redis_version:3.0.7
redis_git_sha1:00000000
redis_git_dirty:0
redis_build_id:f09a0843cc9876c3
redis_mode:standalone
os:Linux 3.13.0-79-generic x86_64
arch_bits:64
multiplexing_api:epoll
gcc_version:4.9.2
process_id:1
run_id:bb0448df5f8ce9d578c7dd73f5b6652368da8911
tcp_port:6379
uptime_in_seconds:593246
uptime_in_days:6
hz:10
lru_clock:3690421
config_file:
# Clients
connected_clients:5
client_longest_output_list:0
client_biggest_input_buf:0
blocked_clients:0
# Memory
used_memory:900400
used_memory_human:879.30K
used_memory_rss:1634304
used_memory_peak:956480
used_memory_peak_human:934.06K
used_memory_lua:36864
mem_fragmentation_ratio:1.82
mem_allocator:jemalloc-3.6.0
# Persistence
loading:0
rdb_changes_since_last_save:0
rdb_bgsave_in_progress:0
rdb_last_save_time:1463287753
rdb_last_bgsave_status:ok
rdb_last_bgsave_time_sec:0
rdb_current_bgsave_time_sec:-1
aof_enabled:0
aof_rewrite_in_progress:0
aof_rewrite_scheduled:0
aof_last_rewrite_time_sec:-1
aof_current_rewrite_time_sec:-1
aof_last_bgrewrite_status:ok
aof_last_write_status:ok
# Stats
total_connections_received:113
total_commands_processed:87811
instantaneous_ops_per_sec:0
total_net_input_bytes:22290687
total_net_output_bytes:1335177
instantaneous_input_kbps:0.00
instantaneous_output_kbps:0.00
rejected_connections:0
sync_full:0
sync_partial_ok:0
sync_partial_err:0
expired_keys:0
evicted_keys:0
keyspace_hits:1
keyspace_misses:6
pubsub_channels:0
pubsub_patterns:0
latest_fork_usec:3636
migrate_cached_sockets:0
# Replication
role:master
connected_slaves:0
master_repl_offset:0
repl_backlog_active:0
repl_backlog_size:1048576
repl_backlog_first_byte_offset:0
repl_backlog_histlen:0
# CPU
used_cpu_sys:824.02
used_cpu_user:331.38
used_cpu_sys_children:0.04
used_cpu_user_children:0.02
# Cluster
cluster_enabled:0
# Keyspace
db0:keys=1,expires=0,avg_ttl=0
Full version of report at the time of discovery can be found here
Now, as a bit of a backgrounder, I used UFW to configure my firewall when I first set up my server. I know that it acts as a “front-end” to iptables, but really didn’t know more than that. I assumed that denying all incoming connections by default, and only allowing on certain ports would be a done deal. Apparently not.
My UFW configuration:
To Action From
-- ------ ----
80/tcp ALLOW IN Anywhere
22 ALLOW IN Anywhere
443 ALLOW IN Anywhere
80/tcp (v6) ALLOW IN Anywhere (v6)
22 (v6) ALLOW IN Anywhere (v6)
443 (v6) ALLOW IN Anywhere (v6)
It turns out that Docker doesn’t play well with UFW at all! This post explains in detail the measures you need to take in order to have Docker play nice with UFW on Ubuntu. Seeing as I never took those precautions my Docker containers with exposed ports were not being protected by any type of firewall. I logged into the database from my desktop, which was totally not supposed to be possible, to see if anyone left me any presents.
I was presented with two keys left of my server (there were supposed to be zero, as I only use this instance for PUB/SUB capability). Both of them were SSH private keys, one was for a root user for a random machine, the other was for a user at some Russian netsec website that I won’t link here. What were they used for? Who knows. Perhaps they were placed there for botnets to query and inject into vulnerable servers so their C2 nodes could get access, or maybe I was compromised much worse than I realize and those were left as a form of digial graffiti from the attackers. Who knows?
Lessons learned
For technologies like Docker, which put complex abstractions over both the file system and network it’s important to understand exactly what each command
or flag you issue does. When I decided to run docker run --name redis -p 6379:6379 -d redis I had no idea that port 6379 would be exposed, and that UFW
would be useless to protect my machine. At least I was able to figure out what happened, if you do even a cursory glance at Shodan there are thoudands
of machines out there that aren’t as lucky!
