This guide will help you understand:
Control fundamentals:
- what is a control
- the differences between:
- a risk owner and a control owner
- controls and treatments
- designing, implementing and evaluating controls.
Evaluation of controls and treatments:
- what control effectiveness means and why it’s important
- how to determine if controls are effective and tips on how to strengthen controls
- the guidance on control effectiveness testing provided in the Victorian Government Risk Management Framework(opens in a new window) (VGRMF) 2020
- how to complete the control effectiveness section of your risk register.
Example
Throughout this guide, we’ll be using examples from a fictional organisation, Welcome Health, to illustrate best practice.
Welcome Health is a 175-bed rural public health service. They employ around 800 staff and service an area covering approximately 120 km radius from the hospital, with an estimated population of 50,000 people.
We’ll introduce two staff members: Anika, Building Manager and Kim, Insurance Manager to bring the theory of control effectiveness testing to life.
What is a control?
A control is something an organisation is currently doing to ‘modify’ a risk. Modify usually means you’re trying to reduce or manage a risk. The purpose of a control is to reduce one or both of the:
- likelihood of a risk occurring
- impact of the risk.
Controls take many forms, including policies, procedures, practices, processes, technology, techniques, methods or devices that reduce a risk. They may be manual (requiring human intervention) or automated (technology based).
Controls typically work in one of three ways:
- Preventative – reduces the likelihood of a situation occurring, such as policies and procedures, approvals, technical security solutions built into a system, authorisations, police checks and training
- Detective – identifies failures in the control environment, such as reviews of performance, reconciliations, audits and investigations (internal or via a third party)
- Corrective – implemented after an event, addresses the root cause or reduce the consequence. They can remediate (e.g patching a system vulnerability), recover (e.g restoring a process or system following a business continuity plan), or respond (e.g coordinated crisis communications with people affected by an incident.)
We learn more about the impacts of these types of risks in Options for Controlling Risks.
Example
Welcome Health’s risk register includes a risk related to service operations:
Risk event
Disruption to critical services and operations
Causes
The inability to access our building due to inadequate fire prevention measures
Direct cyber-attack on our system
Consequences
Damage or loss of building
Financial loss due to costs of managing a cyber incident
IT system outage
There are three controls in place aiming to modify this risk:
Preventative: % of staff who have completed the mandatory fire safety training.
Detective: % of smoke alarms that detect the occurrence of fire that have been tested and are fully effective.
Corrective: Property insurance provides funds to recover from damage caused by fire;
Business continuity plan that has been reviewed and exercised in the past 3 months.
Where do controls fit in the risk management process?
The first steps in any risk management process are to establish the scope, context and criteria (this is your appetite, and criteria for evaluating risk e.g. consequence and likelihood tables) you’re operating in and to identify risks.
The next step is risk analysis. It’s rare that a risk, even a new one, doesn’t already have some controls in place, so it’s at this point that you want to test the effectiveness of those controls.
If the controls aren't meeting or don't adequately modify the risk, your next step is to develop a treatment plan if required (e.g. the risk rating after considering your controls and their effectiveness is not acceptable). These actions when completed become a new, or improved, control.
Ongoing testing of the control effectiveness then becomes an important part of the monitoring and review cycle.
Your controls and their effectiveness will be documented in your risk register.
Check out these resources if you need more information on the risk process:
What is the difference between controls and treatments?
A control is a measure that currently modifies a risk, usually with the aim of reducing or managing it. A risk treatment is a future planned action to address a risk.
The key difference is that treatments are new, and controls are existing. When a treatment is implemented, it becomes a control.
Example
Welcome Health carried out a fire safety audit. The audit found smoke detectors were an older design and had a short battery life. The organisation decided to purchase and install new smoke detectors with lithium batteries. These actions were added as a treatment on the risk register.
Once the actions were completed, having operational and tested smoke detectors was then recorded as a control.
What’s the difference between a risk owner and a control owner?
A risk owner is the person with the accountability and authority to manage a risk, including understanding what the controls are and how effective they are at modifying the risk.
A control owner is accountable for implementing and maintaining specific controls. This accountability may be recorded in a risk register or documented in position descriptions, or in organisational policies and procedures. Control owners may also be responsible for improving controls to increase their effectiveness.
A person might be both a risk owner and a control owner for one or more control, but often the roles are filled by separate individuals.
Example
As building manager, Anika is the owner of the risk, but she doesn’t manage all the controls for it.
One of the key controls for the risk statement is Property Insurance, and it’s Anika’s colleague Kim who manages Welcome Health’s insurance. In this case, Kim is the ‘control owner’ because she is responsible for this particular control: Property Insurance.
Methods for identifying controls
The bow-tie: a method for identifying controls, or the 5-step method
- Bowtie identifies causes and consequences
- 5-step method
Your first act is always to assess the risks to achieving your objectives.
One of the results of that assessment is an evaluation of the risk, which helps you assess what kind of investment you need to make to control the risk.
Your executive team will also have determined your organisation’s risk criteria.
The risk criteria is useful because it sets the boundaries in which you can operate (often quantitatively) and is based on your organisation’s appetite for risk. One of the reasons the Victorian Government Risk Management Framework (VGRMF) requires your responsible body to define its risk appetite is precisely to help you make these decisions.
You may need to consult more widely or escalate decisions to the right decision-making body within your organisation, or even outside if it’s a shared risk or a state-significant risk.
Finally, don’t confine your attention to what’s going on within your organisation. You’re part of a supply chain, in contractual arrangements that expose you to risk. You need to look at those relationships, own the risks and control them. Use the PESTLE tool to help you analyse your external context.
The bow-tie brings it all together
We encourage you to use the bow tie to do your risk assessment. As you can see, it’s a good way to lay out your event, together with its causes and consequences. Use it to identify how you can control the likelihood of the event happening and the severity of the consequences if it does.
Investigating your options
As part of assessing your risk, you also analyse the causes of a possible event, its consequences, and how likely it is.
This analysis will produce the information you need to work out what your options are for controlling the risk. For instance, could you
- avoid the risk entirely by taking another path to your goal?
- remove the source of the risk?
- share the risk with another agency?
- share part of a risk with insurance or a contract with another party?
- make it less likely for the event to happen?
- reduce the consequences if the event did happen?
The international standard on risk management, AS ISO 31000(opens in a new window) : Risk management - Guidelines, refer to these as your options for risk treatment.
When you’re analysing your event, try to step back so you can get a full sense of the risks in your internal and external context. If it’s a significant risk, spend some time working on the scenario. It may be worthwhile to explore a range of scenarios: probable and worst case, and even rare events that would still have consequences the organisation or the state doesn’t want.
Here, we’ll explore risks relating to a cyber threat. We’ll first describe a risk and then set out the range of options for controlling the risk. The aim is to show you what an analysis could look like and how you would consider a wide range of options for controlling the risk. You can then decide what should go into your treatment plan.
Options for controlling risk
So far we’ve looked at how you might assess your options for controlling risk, using the bow tie to analyse how you can control likelihood or the severity of the consequences.
You may not need to do that bespoke analysis though. Many regulatory and accreditation processes build in controls when they require you to demonstrate that you have certain procedures in place. There are examples of this in health care and education.
Procurement, privacy and prudential standards should all be understood as ways of controlling risk—if you comply with the standard, then the risk of financial waste or corruption, for example, is controlled. This is also true for compliance with legislation such as the Climate Change Act and the Modern Slavery Act.
Moving out of the compliance sphere, we can also find examples of voluntary codes and strategies, where someone’s already done the work to validate their effectiveness in a wide range of situations. The Australian Cyber Security Centre’s Essential Eight or AS/NZS ISO/IEC 27001- Information security, cybersecurity and privacy protection — Information security management systems — Requirements are examples of that.
Preventing, correcting and detecting
Another way to look at controls is to look at the structure of risk.
Preventative controls reduce the likelihood of an event happening.
The best way to control a risk is to prevent it from arising in the first place. You might be able to reduce the likelihood of loss or harm to, or close to, zero. Two examples of this, one from everyday life and the other from the workplace, are:
- the design of electric plugs and sockets which make it impossible for a person to touch a live current
- a protocol for releasing confidential information with approval steps designed to ensure that any release for any purpose is approved by the appropriate person in the organisation.
Detective controls pick up the signs that a risk is changing, or an event has happened.
Another way to control a risk is to attempt to detect and report undesirable events or conditions, enabling you to respond promptly and take corrective actions. Examples of detective controls are:
- Audit reviews: Regular audits by internal or external auditors to review financial and operational processes
- System logs and alerts: Using automated systems that log activities and flag unusual actions for review.
Corrective controls reduce the severity of the consequences if the event does happen.
These controls are activated after a risk event has been materialised, aiming to restore operations to their desired state and to prevent the recurrence of similar problems in the future. Examples of corrective controls include:
- A Business Continuity Plan that has been reviewed, exercised and updated in the past 3 months
- An IT Disaster Recovery Plan that has been reviewed, exercised and updated in the past 3 months.
Monitoring changes to your risks
Risk is dynamic. A risk may increase or decrease as a possible event becomes more or less likely. It can also increase or decrease according to a change in the potential consequences. You need to monitor signs of that change.
These signs are your risk indicators.
Your risk analysis will help you to work out what indicators you need to pay attention to, by giving you insight into the causes of events and the factors that make them more likely and their consequences more harmful.
By watching these indicators, you’ll be able to see
- whether your efforts to control the risk are effective
- when you need to escalate a risk that’s approaching a threshold of tolerance.
An evidence-based approach to controlling risk depends on risk and performance indicators. Not only will it help you achieve your objectives, it’ll help you demonstrate, in economic terms, the value of managing risk effectively.
What is control effectiveness and why is it important?
Control effectiveness is the term used to describe how well a control is reducing or managing the risk it is meant to modify.
The more effective a control is, the more confidence you have the risk is being managed as you expect. A control is more effective when it is highly:
- relevant (it’s designed to address the intended risk)
- complete (it addresses most/all the risk)
- reliable (it operates as expected)
- timely (it operates at the right time and reacts quickly enough).
Understanding how effective your controls are will assist you in planning and prioritising risk management actions and making informed decisions.
What is control effectiveness testing?
Control effectiveness testing involves regular review of your controls, to ensure they’re designed correctly and are effectively reducing or managing risks as expected.
Note that you are evaluating whether the established controls (preventive, detective, and corrective) within your organisation are operating as intended and are effective in mitigating identified risks to acceptable levels.
The purpose of control effectiveness testing
- Assess adequacy and effectiveness: To determine if controls are adequate to address specific risks and if they are effective in either reducing the likelihood of an event occurring or minimising its consequences should it occur
- Identify weaknesses and gaps: Testing helps identify weaknesses or gaps in the control environment, providing an opportunity to strengthen controls before failures occur
- Compliance verification: Your organisations must adhere to various regulatory requirements and standards. Control testing ensures compliance with these mandates by verifying that controls meet or exceed required standards
- Support continual improvement: By regularly testing controls, you can make informed decisions about where to make improvements or adjustments, supporting a continual improvement cycle in risk management.
Control effectiveness testing may be more suited for organisations that have stable control environments, mature risk management frameworks and resources available to do the testing. You may choose to test some controls more often than others, or to prioritise testing based on factors such as internal audit findings.
The VGRMF includes control effectiveness testing in its guidance on good practice risk management, however it’s not a mandatory requirement.
Risk and/or control owners are usually responsible for testing control effectiveness and ensuring the control is working as intended.
Example
Kim is new to her role and wants to understand the effectiveness of the control: Property Insurance.
She read the policy documents and found that she had some questions about coverage for fire safety equipment at a property that Welcome Health owns. Kim contacted VMIA for clarification about the extent of the coverage, which gave her confidence that the control was meeting Welcome Health’s needs. Regularly reviewing policies to ensure they meet current/changing business needs is a way to test the effectiveness of a control.
So, how do you test for control effectiveness?
Testing for control effectiveness includes:
- Understanding the control’s purpose
- Gathering evidence
- Evaluating, and
- Planning treatments and update the risk register.
Find out more about each of these steps in 'How do I test control effectiveness'.
Weighing up the costs of control
Controlling and monitoring risk, like all management activities, comes with a cost. There are two ways to look at this:
- weigh up the cost against the benefit of achieving your objective
- look at the opportunity cost of spending money on controlling the risk, rather than something else of value to the organisation.
Costs and benefits
The benefit you’re seeking comes from achieving your objective. So, your question here is whether the cost of controlling the risk you need or want to take to achieve your objective is worth it.
For example, is it worthwhile to invest $500,000 on building upgrades when the whole organisation will be moving to another office within 12 months?
Opportunity costs
The other concept to bear in mind is opportunity cost. Your organisation’s resources are finite. If you decide to spend money on controlling a risk so that you can achieve an objective, then that money isn’t available for other work.
That needs to be a conscious decision. When you’re making decisions about how to control your risks, you should always ask yourself whether that puts other objectives at risk.
For example, if you didn’t spend that $500,000 on the building upgrade, that money would be available for an upgrade to the health and well-being program and other culture initiatives.
This is how you demonstrate, in economic terms, the value of managing risk effectively.
What is a control library?
A control library is a central list of all your organisation’s controls. Control libraries are best suited to larger organisations, where there’s a critical reliance on operational processes that need to be well documented and regularly monitored.
It may contain:
- a list of controls
- a description of each control
- the risk/s a control aims to modify
- the effectiveness of a control
- the control owner
- categorisation into preventative, detective, or corrective
- categorisation into key controls or non-key
- categorisation of general type (IT, manual, environment).
A control library allows you to see all controls operating within your organisation and their effectiveness in one place. It allows you to view controls that are in place for other business areas, so you can consider using or adapting them to address other risks.
For some organisations, control libraries exist as databases or modules within their governance, risk and compliance applications/systems, including safety, IT and finance. Control libraries are sometimes used by auditors to identify and test the key controls relating to an audit area.
Take a look at our control library example:
Updated