How to design and lead tabletop exercises (TTXs)?

When it came to tabletop exercises (TTXs), we didn’t have to look far to find experts with a wealth of experience in designing, planning and facilitating TTXs of various scales and scopes. We invited two of our capacity-building experts, Aistė and Tadas, to share their insights on designing realistic yet engaging exercises, measuring their impact, and organising large, multi-organisational activities.

Dr. Tadas Jakštas, cyber capacity building team lead at NRD Cyber Security

The first exercise that I led in 2017 was a NATO-led exercise in Ukraine. The aim of this exercise was to assess Ukraine’s readiness to deal with hybrid threats to Critical Information Infrastructure (CII). The exercise covered areas such as international law and strategic communications. My role involved preparing cyber syndicates, which entailed drafting various scenarios and other materials. Over 200 participants from different sectors were involved. I was the lead moderator for the cybersecurity syndicate.

In 2019, I was involved in the preparation of an energy sector exercise in Lithuania, which enabled operators from all three Baltic countries to identify cyber threats and test their ability to handle them. I also participated in a regional exercise in Odesa in 2021 to test regional cooperation measures, particularly with Ukraine’s neighbouring countries, such as Georgia and Moldova.

Aistė Aurelija Azbytė, cyber capacity building expert at NRD Cyber Security

My experience with TTXs began when I participated in an exercise organised by the Atlantic Council. This exercise involved representatives of various ministries and government agencies, and exposed me to high-profile military defence issues and discussions. For example, I learned how Lithuania is preparing to receive allied forces.

While working for Lithuania’s NCSC, I co-organised an exercise in Georgia as part of an EU twinning project, in which CII organisations had to consider how they would respond to cyber threats, and how they would mitigate and handle cyber crises.

I was also responsible for organising a strategic-level exercise called ‘Cybershield Startex’, which dealt with a cyber-attack on the energy sector. This exercise was carried out in cooperation with the National Crisis Centre under the Government of Lithuania.

1. What would you say is the recipe for designing a realistic and engaging TTX?

Tadas: First and foremost, you need to understand the objectives. Are we testing a hypothesis or an existing scenario, or do we want to design a plan? This greatly affects how you design scenarios, injects, vignettes, etc. It is also important to discuss the level of the exercise at an early stage – how technical or strategic should it be?
Of course, as the exercises are designed, run and participated in by people, it is crucial that appropriate stakeholders take part and are assigned specific roles. The moderator is under a great deal of pressure – they need to understand the context, be able to ask questions and provoke reflection and commentary as needed, while also listening and facilitating discussions and steering them towards the point. The moderator should also avoid trying to outsmart the participants. Large exercises require more than one moderator, and if the exercise is regional or multi-country, having both local and international moderators with different perspectives is especially valuable.
Lastly, validation plays an important role in such cases. It is important to check the facilities and logistics months in advance, as it may be easy to alter something at the time of the check-up, but extremely difficult to sort things out on the spot during the tabletop exercise.

Aistė: I would also add, that to make it realistic yet engaging, the scenario should combine fear and information. It needs to reflect what could happen to make it relatable for the participants. This is why the tabletop exercise should be co-created with someone from inside the organisation who can provide first-hand information.

Tadas: I can only second that and stress the importance of involving local experts in the design of scenarios, injects and vignettes. For example, I was involved in an exercise in Ukraine where Ukrenergo, the country’s largest electricity transmission system operator, provided insights into their electricity supply setup. This context enabled us to make the TTX scenario more realistic for the country. In another exercise that I co-organised, cybersecurity was only one part of the TTX, as there were five different syndicates. Once the syndicate groups started sharing information, it became much easier to identify the scale and scope of the hybrid threats facing the country.

Aistė: Another small but important detail is that housekeeping rules need to be set at the beginning, e.g. do not fight the scenario.

2. What should be considered when organising large, multi-organisational exercises? What are the pros and cons?

Tadas: Proper planning is very important if the exercise is big on a scale. Making it realistic takes a lot of time and effort, and requires a lot of planning, organisation and administration. In certain cases, e.g. NATO, there is usually strict guidance on preparation. A structured approach really helps you to focus on what is important, because in large exercises there are so many things to consider and address. From a moderator’s perspective, discussions are much more comprehensive, so a more experienced and prepared moderator may be needed.

Aistė: Large exercises can be extremely useful, creating added value by raising awareness, igniting cooperation and improving understanding of roles in case of certain scenarios. It is an opportunity for various stakeholders to gather in one room or facility and dedicate their time and attention to the matter. So in order for the momentum to last, it is important to hold a debriefing session promptly to capture fresh insights. Over time, things can become muddled and it can be difficult to identify the most important aspects.

3. Is there a way to measure impact the TTX has had?

Aistė: Your evaluations should be based on the objectives you had for the TTX. Feedback from participants is a good source of information, as is an assessment of whether people have learnt about their role in a specific scenario and the subject of the TTX. Have the stakeholders become aware of anything that could speed up processes later on? The TTX on providing logistical support to the allied forces that I mentioned before, for example, clearly demonstrated the need for awareness and support throughout the entire government, not just individual ministries. I was personally surprised that the Ministry of Environment was a stakeholder in this matter. Also, TTXs can sometimes inspire changes in legislation – this has happened in the energy sector in Lithuania, for example. However, it may take a few years for these changes to come into effect.

Tadas: I also agree that evaluations can be conducted by gathering key takeaways from participants. I would also add that having observers who watch and listen to proceedings and then provide feedback afterwards is incredibly valuable. These observers are usually independent experts in certain fields. They provide insights to participants and support the moderator by giving them feedback on their performance.

4. Who is usually responsible for identifying the need for TTX within an organisation?

Aistė: In my experience, it is usually somebody who has identified a specific issue (a bottom-up approach), but the involvement of people at various levels is crucial.

Tadas: I would say it depends on the objectives. If the organisation wants to test its cyber incident response plan, it may be the CISO or SOC/IT director. If it is a more strategic, high-level scenario in question, such as a national cyber crisis, it may be the head of the National Crisis Management Bureau. For regional exercises testing ally preparation for cyber threats, it could be NATO.