Top tools and resources for running a capture the flag competition

To stop the cyber-attacker, you must think like the cyber-attacker. This is a skill that needs practice, and to get that practice, hackers created capture the flag (CTF) competitions, where they compete to p0wn servers and gain cred.

It wasn’t long ago that such activities were of questionable repute and legality. It’s all out in the open and very respectable these days, even if participants adopt an alias and play Bond villain.

CTFs for enterprise security staff are a win-win for the white hats. Security personnel learn new techniques, practice tackling challenging scenarios, and network with others in the security community. It doesn’t stop there.

Bobby Kuzma, director, cyber threat strategy and enablement for IT automation and security software vendor HelpSystems, says “I see a decent number of enterprises actually use CTFs as part of their community outreach and recruiting strategies. It helps get people, especially students, excited about cybersecurity, and identifies promising non-traditional candidates.”

You can find many lists of CTF resources with a simple web search; a large number of them are on GitHub. Some of the resources are for building CTFs and some help those who are competing, and there is a good deal of overlap. A few examples include awesome-ctf, AnarchoTechNYC and zardus.

The largest set is hacking resources. All hacking resources, defensive and offensive, are CTF resources: source and binary static analysis, packet capture, debuggers, decompilers, heap visualizers, hash crackers, image editors and network scanners. All security experts have their own sets of favorite tools, but a CTF may challenge them to find new ones.

One personal favorite resource of mine is Didier Stevens and his tools. Didier’s original specialty is tools to analyze PDFs, Microsoft Office documents and other complex data files, many of which are used to perpetrate attacks. His collection is much more varied now. You can find them all in his GitHub repository. They are invaluable for examining and creating malicious files.

Types of capture the flag events

I’ll get to other tools that are more specifically geared toward CTF, but first, let me review the two main styles of CTF: attack-defend and Jeopardy-style.  

Attack-defend In an attack-defend competition, there are two teams, each with a computing environment, which may be as simple as a single server. Each team tries to attack the other’s systems and defend their own from attack. Each system contains a number of informational flags that the attacker tries to find and capture. This is where the name “capture the flag” comes from (that and the traditional outdoor game).

In such a scenario, defenders need to do all the things they would want to do on their own real-world servers: Patch all software vulnerabilities, even the obscure ones; leave open only the very necessary services through the firewall; make sure all passwords are strong and that accounts are given the least privilege necessary; and so on.

The attacker, on the other hand, uses penetration techniques to gain privileged access to the server. Certainly, if the attacker can get root access the game will soon be over, but depending on the applications and services involved, more limited attacks may be sufficient.

Jeopardy-style Jeopardy-style tournaments have any number of teams and a Jeopardy-style board with challenges worth different numbers of points. If a team takes a particular challenge and finds the flag, they submit it to the scoring system, get the points, and move on to the next challenge. When the clock runs out, the team with the most points wins.

Because they are much easier to set up and administer, Jeopardy-style competitions are far more common than attack-defend.

Ox002147

King of the hill In a King-of-the-hill event, each team tries to take and hold control of a server. When the clock runs out, the team that held it longest is the winner. It is a variation of the attack-defend CTF.

Why would you choose one type of contest over another? Kuzma says that “Jeopardy events are good for building problem solving skillsets.” King of the Hill events are excellent for strengthening incident response, collaboration, and planning. “Ultimately, any type of training that puts a security pro outside their comfort zone is a benefit.”

Where to find capture the flag events

Groups all over the world hold open competitions all the time. One of the main places these events are organized is on the site CTFtime. The large majority are Jeopardy-style. Of the 152 events in the archive for 2018, 16 are attack-defend, 135 are Jeopardy-style.

If CTFtime is the ESPN of CTF, then the Super Bowl of CTF is at DEF CON, the annual hacker conference in Las Vegas. The winners of the DEF CON 26 CTF in August 2018 were the DEFKOR00T team. The complete archives of past DEF CON CTFs are available on their servers. Another famous CTF happens at the annual NorthSec conference in Montreal.

The big events like DEF CON are held in one location, but most CTFs are online and worldwide. The National Cyber League (NCL) organizes Jeopardy-style competitions aimed at high school and college students. There are defined seasons and contests.

Most of the events on CTFTime are held by small groups of security enthusiasts, but not all. Late 2018 saw the Trend Micro CTF 2018, with the final in Tokyo, which includes a king-of-the-hill competition. On the other hand, on April 20, 2019, a six-day competition run by the Computer Security Club of the Thomas Jefferson High School for Science and Technology in Fairfax, VA will be held. Yes, that’s a high school. The CyberPatriot competition is held by the US Air Force for high school and middle school students.

CTF events at major security conferences like DEF CON are high profile, but many enterprises organize their own. It is a genuinely good learning tool and an exciting switch from the often-boring day-to-day work of enterprise security.

Building your own CTF event

So, how do you make your own CTF? As an enterprise, used to dealing with professionally developed products with polish and support, you may be disappointed with what you find. There are very few options for a turnkey CTF, but you can collect endless bits and pieces and arrange them into a unique and challenging contest.

The closest thing to CTF-in-a-box is the OWASP Juice Shop. OWASP (the Open Web Application Security Project) is an organization of security professionals who design tools and guidelines to help developers and other IT professionals make secure applications and has chapters worldwide.

The Juice Shop is a fictional web-based store that sells juice, t-shirts and other things, the details of which are unimportant. What matters is that the site is replete with vulnerabilities of just about every known kind. The site is customizable so that you can brand it as you wish and change the products to be what you wish. OWASP distributes it in a variety of forms, including as a Docker image, and it runs in a single server instance.

OWASP

Juice Shop also includes the scoreboard and account management necessary to run a competition.

Capture the flag frameworks

These are a few of the most popular CTF frameworks as well as some that are a bit more obscure. CTFd is a CTF platform used widely by security vendors, colleges and hacking groups. It includes the scoreboard and other infrastructure of a contest. You just add the actual challenges, which are the puzzles solved by users, and their scores.

Other major frameworks include:

• Facebook’s CTF framework
• iCTF from the UC Santa Barbara Computer Security Lab
• HackTheArch
• Mellivora
• NightShade
• LibreCTF
• picoCTF

Capture the flag tools

Google holds some significant CTFs. It has not released its entire framework, but it has released its scoreboard code and most of the challenges.

The list of helpful tools is long. Here are a few:

• Security Scenario Generator (SecGen) generates semi-randomized vulnerable virtual machines.
• mkctf creates challenges in a predefined format for input into a framework.
• Damn Vulnerable Web Application is an open source PHP/MySQL web application built to exhibit known and unknown vulnerabilities. The user chooses a vulnerability (e.g. SQL Injection) and uses the UI to invoke it. The DVWA has no amusing front-end like the Juice Shop, but sometimes the simple route is the best.

Where to find capture the flag write-ups

Many of the best resources, particularly for Jeopardy CTFs, are the records of CTFs past and post-mortem write-ups by participants describing them. If you look around you can find write-ups on CTFs that describe the challenges and how they were solved. Find enough of these and you may be set. You can find a large archive of write-ups, as well as tools for making write-ups, on this github CTFs page.

The write-ups are detailed enough that you can, with not a lot of work, change things around to make the challenge your own. The main problem with the write-up archives is that many of them are lists of challenges for which the write-up is “to do.” Another downside with many write-ups is that many of the authors don’t write well.

Running a capture the flag event in the public cloud

Because of the ephemeral nature of CTFs, it’s tempting to run them in a public cloud where you can allocate the resources for them and deallocate afterwards, paying only for what you use. You can do this if you are careful and play by the rules.

Amazon Web Services (AWS) has rules for penetration testing. You have to fill out a permission form and test only against a set list of services. You can’t use low-capacity instances. Follow the rules and wait for permission, because Amazon looks for the sort of behavior users exhibit in CTFs and may block it.

Microsoft also has strict rules for pen testing on Azure, but they do not require preauthorization to do it.

Google also requires no preauthorization, just that you conform to the Google Cloud Platform Acceptable Use Policy and Google Cloud Platform Terms of Service.

A CTF is likely to be a popular event with employees, more so than conventional training, and perhaps more useful. At a time when many security positions go unfilled, a CTF can be a valuable recruitment tool, finding you to the most skilled candidates in an objective way. Look at them as a tool to maximize your team’s skills and the fun part just comes for free.