Early KS1 to KS2 Progress Analysis

• Your LA and schools will benefit from access to the earliest available and most robust provisional KS1 to 2 progress scores at school and LA level - much earlier than the release of DfE data.
• You can inform your schools and senior management, during the summer term, allowing them to understand the implications of the key progress accountability measure for their schools and to plan for the new academic year.
• The LA and schools will know in advance which schools are likely to be assigned a coasting designation or to be below floor standards.

Introduction

This guide is intended to help you understand how to carry out Early KS1 to 2 progress analysis within Nexus. It includes hints and tips about how to ensure your LA import your KS2 data as quickly as possible after the release of the KS2 checking files.

You can undertake many of the steps ahead of the submission period, this reduces the workload at the key point when KS2 results are issued, and allows the multi-LA aggregation to build up faster to the benefit of all LAs.

Background

Following the implementation of the new KS1 to 2 progress measure (Value Added Model) by the DfE, three local authorities (Wigan, Calderdale and Bradford) collaborated in 2016 to collect aggregated results from as many English LAs as possible in order to try and determine the progress scores for all the LA and schools involved.

This approach was adopted by NCER and brought inside Keypas and Nova for the benefit of all LAs in England. LAs can now load in their KS2 checking file in the second week of July and run KS1 to 2 Progress analysis for their LA and schools during the summer term. Early KS1 to 2 Progress results in 2017 were accurate enough to release to schools. The NCER will keep the membership up to date with any live issues via the Nexus Forum.

In any given year, as more KS2 checking data is loaded into Keypas, the progress scores will continue to change and become more robust. Local Authorities will wish to take care to ensure that the dataset has stabilised before making use of the progress scores in Nova. Please view the forum post on “Fullness” to see how much data has been loaded into Keypas.

In addition, reports indicate the number of pupils, schools and LAs that have contributed to the emerging national averages for Early Progress analysis. Users should regularly run these reports, checking these numbers and making an informed decision on when to make use of the results, balancing completeness and accuracy with utility to school leaders before the summer holiday.

Keypas & Nova Process

Matching The Data

Keypas already holds KS1 results for the large majority of pupils who will undertake KS2 tests and teacher assessment in any given year.

The DfE will provide each LA with a KS2 checking file from Primary Assessment Gateway (PAG); in 2022, this is expected on Tuesday 5th July.

The ideal is that all LAs in England should aim to import their KS2 data into Keypas within a couple of days of release. This is used not just for Early Progress but also for attainment reporting.

During the import process, Keypas will try to match each imported KS2 pupil to a KS1 pupil for the relevant historical cohort, this is a cross-border match based on UPN, date of birth and sex.

Where all three identifiers match perfectly then the records will be linked, otherwise no prior attainment data will be available for that pupil. LAs have the opportunity to load in any missing KS1 results for unmatched pupils (see “Filling the Gaps” section of this guide).

Once matched, the pupils KS1 data is copied for use in progress calculations, therefore manual modifications to KS1 data via Keypas grids, are not used in progress calculations.

The matched data is visible to users in the KS2 “Data from DfE” > Imported Data > Pupils grid in Keypas and should be sampled to check for accuracy in the matching process before any reports are issued.

Missing KS1 Pupils

Any gaps in the KS1 prior attainment (or errors you are aware) can be filled by a manual import of KS1 results. This is likely to be where a pupil UPN number has changed or there was any discrepancy in the stated date of birth or sex that prevented the three-point match mentioned above from occurring.

The Pupil Calculations

After importing the KS2 checking data, the pupil’s prior attainment values will be used to automatically calculate their KS1 Average Points Score (APS) and the prior attainment group that puts them in.

Until the current year prior attainment groups are published by the DfE, Keypas will use the previous year groups and thresholds. For some prior attainment groups this may differ from the current year versions as they can be composed of such small numbers of pupils. NCER will indicate when these have been updated in a forum post and reports should not be made available to schools until these have been updated.

Once pupils are allocated to a prior attainment group, they are ready to be allocated their progress scores. These scores are based on the NCER emergent scaled scores for data already loaded into Keypas.

The progress score is best explained as the difference between a pupil’s scaled score in a subject and the national average score for their prior attainment group, which is why the ‘fullness’ of that early national data is an important consideration. Both the national averages and the impact of those on individual pupil results are regularly recalculated during the emerging data period. In the initial weeks this will be done hourly and then daily once the data had stabilised.

In addition to the progress scores, standard deviations are also calculated which allows reports to display the 95% confidence intervals alongside the progress scores.

Steps you need to take for your LA

Requirements: The process described below requires access to data from Keypas, Primary Assessment Gateway (PAG), Key To Success (K2S) and the ability to use MS Excel for VLOOKUP or MS Access queries.

TIMINGS

OPTIONAL - ONCE JANUARY CENSUS FILES FOR ALL SCHOOLS (INCLUDING ACADEMIES) ARE AVAILABLE

Prepare KS1 data for all Year 6 pupils on your January census. This will allow you to respond quickly to any pupils that cannot be matched from the KS2 checking data back to KS1 Keypas results.

• Download the Keypas KS1 data for your LA from four years before (e.g. the 2019/20 KS2 Year 6 cohort will have sat KS1 as Year 2 in 2015/16). You will also make use of this data in step 2 below.
• Determine which pupils in Year 6 do not have KS1 data available and take steps to secure the missing information (see ‘Filling the Gaps’ below).

Please note that it is not necessarily the case that a number of pupils are ‘missing’ as their KS1 data may have been submitted by other Local Authorities, but preparing on this basis will save effort in the key reporting period later.

JULY – KS2 CHECKING FILES RELEASE DAY

Set aside time to undertake the following processes as early as possible after the release of the KS2 checking data.

• Download the KS2 checking data from Primary Assessment Gateway for your Local Authority.
• Optional - Cross reference the KS2 data with the KS1 data you have prepared in step 2 using MS Access (linked tables) or MS Excel (VLOOKUP) based on UPN numbers. This is to identify new starters after the January census.
• Any pupils that exist in the KS2 checking data but not in the KS1 data you have available may need to be checked in Key 2 Success if they do not match once imported into Keypas

In Keypas:

• Import your KS2 checking data as per the Keypas User Guide.
• Import your contextual data (census files) as per the Keypas Census Import Guide
• If DfE have already made it available, import the pupil premium data for the current year as per Keypas Pupil Premium Data guide. In Keypas, check the KS2 (from DFE) > Imported Data > Pupils grid and sample records to ensure that the matching has occurred accurately.
• Import KS1 results for any pupils with gaps in prior attainment that can be determined or to overwrite any incorrect values you are aware of. The template file for this import is available on the KS2 from DFE (KS1 Prior Attainment) Keypas import page (see ‘Filling the Gaps’ below).

JULY – IN THE DAY(S) AFTER CHECKING FILE RELEASE DAY

• Wait for the data to stabilise and indicate that enough LAs are covered to provide a viable early national average. Ideally this will be around 140+ LAs within a couple of days of the release of the KS2 data.

In the case of early release of early progress data to schools and LA colleagues, data teams should be clear with schools and management as to the limitations of that data (even as an Early Data set) and return to the reports over the next several days as that will improve with time.

• Release appropriate reports to schools, school improvement and management. Any communication with schools should stress the fact that the data is early and unvalidated and may not match DfE figures released in the SFR. Early-national pupil group data should not be made available on external websites or social media for the same reasons. In 2020 the first cohort who undertook KS1 under the revised framework without levels will take KS2 SATS and - at the time of writing - there is no clarity from DfE on how progress will work. Caution may again be required in publishing early progress reporting for that year.

Filling the Gaps

PUPILS NOT IN YOUR LA KS1 DATA WHO ARE ON JANUARY CENSUS FOR YEAR 6

Some pupils will not match when the KS2 checking data is loaded into Keypas. There are
several reasons this may be the case:

• The pupil has no prior KS1 attainment and will be excluded from progress calculations
• The UPN number has changed since KS1
• Their date of birth or sex was incorrectly recorded in KS1 data

For pupils affected by cases 1. and 3., you can take steps to correct this omission. It is important to do so because – particularly in small schools – one pupil can make a large difference to progress scores (positive or negative) and the overall accuracy of reporting for that school will be adversely affected if this is not rectified.

As above, you can prepare your KS1 data for all pupils several weeks ahead of the import process, saving you time during the key assessment results period. Most of the pupils will automatically match in Keypas but you can discover the identity of most of the potentially ‘missing’ pupils based on your January and May census returns for Year 6.

By extracting the Year 6 UPN, name, sex and DOB details for all pupils in your schools from the census, you can check whether they have matching KS1 records within your own LA by also extracting (and cross referencing in Access or via VLOOKUP in Excel) the KS1 records for the same cohort of pupils from 4 years earlier.

NB: Pupils who exist in the census returns but not in your KS1 data from four years previous will be potentially missing their KS1 starting point when the KS2 results are loaded in.

Knowing this, you can then:

• Use Key 2 Success (K2S) to try and determine what the KS1 data is. You will need to input the UPN initially to try and make a match via a block search. It may be that some pupils still do not match and therefore a name/sex/DOB search may be required to find the accurate match.
• Retain the KS1 information found for loading to Keypas once the KS2 data has become available.
• Update the example file ‘Example KS1 Prior Attainment.csv’ with the KS1 data that needs to be loaded and hold this until you are ready to load it as step 8 of the above process. The example file is available on the KS2 from DFE (KS1 Prior Attainment) Keypas import page.

PUPILS IN YOUR KS2 CHECKING DATA THAT MAY HAVE JOINED AFTER THE JANUARY CENSUS AND DO NOT MATCH ANY LA KS1 DATA

You may need to repeat the process above from checking in Key 2 Success.

ITEMS TO NOTE

It is possible that you will still have some pupils without a KS1 starting point. In the official DfE figures it is possible that these may be included. Therefore the missing pupils form a part of your caveats to schools around why there may be differences.

Please Note: If you need to re-import your checking file then you will also need to re-import your KS1 missing pupil data, as this data will be wiped by the re-import process.

RUNNING REPORTS

Early progress reports are mirrors of the standard KS1 to 2 Progress reports in Nova (though annotated to reflect their provisional early status and including the numbers of pupils, schools and LA). All of these reports will be suffixed with (Keypas) so they can be separated from the standard KS1 to 2 Progress reports that become available later in the year. They can be run by LA users through Nova but are not currently available to schools via their ‘LA > Reports’ option in Perspective Lite.

You will therefore – once you judge the data is robust enough to do so – need to produce and distribute the reports for your schools.

You may wish to add your own annotations to those reports for the purposes of publishing them, but you will certainly want to make sure that schools and LA colleagues understand the unofficial status of the reports and any issues affecting the accuracy in that year. You can update the footnotes of reports in Nexus by going to Nova > Admin > Preferences.

Help Section of Nexus