Click Questions to see example responses, some of which include embedded links to reference sources.
Compliance or Quality Assurance (‘QA’) checks effectiveness of deployment of internal process and procedure (e.g. Is a consistent approach applied to the identification of and response to risk, via the adoption of senior management agreed protocols and standards).
Testing focuses on assessment of the documented framework, oriented towards checking completeness of coverage (or conducting a gap analysis of existing content), such as:
Testing focuses on process validation and identifying enhancement needed, if any, to improve process design and output quality. Testing often uses a sample-based approach, oriented towards verifying a population of previously completed activity, such as: a collection of:
Testing a reasonable cross-section or sample of completed work (or workflow elements) also provides a more holistic view on employee understanding, awareness and the effectiveness of their adoption of internal process and procedure, via a combination of:
Whereas QA relates to independent persons testing activity previously completed by a team or function, Quality Control (‘QC’) is undertaken as part of a team or function’s workflow prior to sign-off of a completed process. QC might be the last process element used to determine whether activity has been completed, or if rework is required before approval or sign-off of a deliverable. QC is aligned to inspecting individual cases to check and ensure, or validate, that activity completed is consistent with applicable requirements and to an acceptable quality.
QC validation (often referred to as ‘4-eyes’ checking) focuses on ensuring that pre-defined requirements to produce a deliverable have been completed or, where found to be incomplete, the necessary remedial work is undertaken before sign-off/finalisation.
QC is usually completed as part of business-as-usual (‘BAU’) activity, possibly by a team leader or someone within a BAU team with responsibility for conducting 4-eyes checks.
Quality Assurance (‘QA’) | Quality Control (‘QC’) |
---|---|
Focus on systemic assurance in quality achieved across a population (e.g. collection of customer files). |
Focus on ensuring fulfilment of required quality at a tactical level (e.g. individual customer file). |
Aims to prevent systemic defect, via independent check & challenge of BAU activity. |
Aims to identify and remediate tactical defect as part of BAU workflow. |
Assess overall quality and consistency of procedure application – Verification. |
Check conformity with applicable procedure – Validation. |
Does not involve repeating detailed deployment of a process or procedure. |
Always involves a level of detailed validation (or repeating) of how a process or procedure is applied. |
Preventive tool to identify systems and controls risk in the operational environment. |
Corrective tool to identify issue/anomaly prior to finalisation of a BAU activity/deliverable. |
Proactive measure of effectiveness of policy and procedure deployment across a functional area. |
Reactive measure of policy and procedure deployment, as applied in a BAU process or to an underlying activity. |
Often involves end-to-end or broad-based review of how procedure is implemented in a process life-cycle. |
Often involves detailed validation of procedure adoption in process segments. |
Findings and outcomes inform senior management awareness on systems and controls risk. |
Outcomes inform team/function management on adherence to process and procedure. |
The ‘three lines of defence’ is a non-prescriptive model applied in the internal control framework of many firms regulated by the Financial Conduct Authority (‘FCA’). Formality of and responsibility for leading a defence within a regulated firm's framework, is considered by the FCA when assessing firms functional segregations and reporting structures (e.g. as part of an FCA visit).
The arrangements within a firm to manage risk in business-as-usual ('BAU') activity (e.g. customer-facing operations, plus middle and back-office support):
Supported by the advisory and monitoring functions of Risk Management and Compliance, and often via a delegated committee/functional framework, 2LOD oversees effectiveness of implementation of the internal control framework:
Independent assurance and executive challenge:
Compliance monitoring is the application of quality assurance testing on the day to day activities of the business. The Compliance Monitoring Officer (or Team in larger firms) is usually an independent function within 2LOD, which provides reports of review findings and recommendations to senior management (e.g. following testing of effectiveness of 1LOD’s application of internal controls).
Review activity undertaken as part of an overall Compliance Monitoring Plan (‘CMP’), should be oriented to the structure and risks facing the business, or business areas to be reviewed. CMP content may comprise a library or blend of:
CMP review planning should:
Artefact development for each review might ordinarily cover:
Findings of CMP review activity should:
An appropriate sample is one which provides confidence that findings of a review of the sample selected will be representative of the total population, within an acceptable level of confidence or margin for error - For example, if you use a confidence interval of 5 and 56% of the sample reviewed identified an issue (say ‘X’), you can be fairly sure that if you had tested the same point in the wider population, between 51% (56-5) and 61% (56+5) would have identified the same issue (i.e. ‘X’).
The relevant period to be covered by QA testing is defined in a firm’s risk-based approach. The simple model illustrated in the table below suggests verification is conducted on all higher-risk customers on-boarded in the month, with other customer risk classes covered by a reducing ratio of randomly selected cases:
Risk Class | Number | Sample | % |
---|---|---|---|
High | 4 | 4 | 100% |
Medium | 35 | 7 | 20% |
Low | 20 | 3 | 15% |
Simplified | 8 | 1 | 10% |
Example factors relevant to sample selection:
This FAQ is not intended to be a briefing on statistical analysis, but notes that achieving a higher degree of confidence (i.e. of sample findings being more representative of the total population), a large sample size may be required. However, a risk-based approach provides opportunity to balance effort needed with the potential for risk and an acknowledged margin for error.
Example online tools to aid sample size selection include*:
* SYSC326 is not associated with providers of these example tools, nor do we endorse their functionality over and above others which may be openly available. Links are provided for information purposes only, with any decision to use them being entirely at your own risk.
It is important to be clear as to where ‘Key risk’ is identified and where risk ownership resides (or should reside), particularly when remedial activity may be needed to provide a fix or implement a compliant solution. This may occur when, for example, material non-conformance with a regulatory requirement has been identified, or where it is not possible to demonstrate conformance with a regulatory requirement (e.g. the Money Laundering Regulations require firms must apply enhanced customer due diligence measures and enhanced ongoing monitoring (in addition to customer due diligence measures), to manage and mitigate the risks arising in certain higher risk cases).
In addition to the risk of prosecution for non-compliance with a legal requirement, a regulated firm’s inability to respond to relevant higher risk cases could be a red-flag alert to a regulator, about the particular firm’s approach to financial crime risk management, its internal culture and/or the firm’s senior management arrangements.
Where possible, review findings should be clear on the how and where non-compliance is identified, such as:
As well as the ‘how’ and ‘where’, senior management need information to understand the risk-significance (or materiality) of any non-compliance identified. One way of doing this is to apply a rating or score to indicate relative ‘Significance’ of an issue, where: Priority Risk Rating = Consequence x Likelihood.
Where a firm already utilises an effective risk-rating framework the same approach should be adopted for consistency. Where an in-house approach does not exist or is in early stage development, the simple 3-element model below might be useful to incorporate or amend for use within a firm.
Consequence Rating
Rating |
Assessment |
Business Impact (examples) |
1 |
Extreme |
|
2 |
Very High |
|
3 |
Medium |
|
4 |
Low |
|
5 |
Negligible |
|
Likelihood Rating
Rating |
Assessment |
How likely is this event to occur? |
1 |
Almost Certain |
Highly likely, this risk/event is expected to occur. |
2 |
Likely |
Strong possibility that a risk/event will occur and sufficient reliable trend analysis or historical data (e.g. internal incidence data, external reporting by regulators’, etc.) supports the assessment. |
3 |
Possible |
Risk/event may occur at some point in the next [1 - 3 years], although reliable historical data is not currently available to support the assessment. |
4 |
Unlikely |
Not expected, but a remote/slight possibility the risk/event may occur. |
5 |
Rare |
Highly unlikely, but might occur in unique circumstances, or if a material change occurs in the business or BAU activity. |
Priority Risk Rating
Rating |
Assessment |
Priority recommendations or proposals, etc. |
1 - 2 |
Severe |
|
3 - 4 |
High |
|
5 - 7 |
Significant |
|
8 - 14 |
Moderate |
|
15 - 19 |
Low |
|
20 - 25 |
Trivial |
Factors influencing QA testing include:
Factors influencing QC capability include:
Findings should be measurable and used to inform senior management view on, whether:
Findings should provide team/functional management with a view on, whether:
For more on our Compliance Assurance Services - See Compliance Assurance.