Overview

In order to streamline the process of getting verified scores into applicant records in Slate, an integration has been created to transfer records from the ETS Data Manager to the SPS Slate instance.

Stakeholders

SPS Admissions

Special Considerations

Dependencies

In addition to having Node.js installed, the following libraries need to be installed on the server that will host this application:

• npm install jsdom
• npm install easy-soap-request
• npm install moment
• npm install os

Feed Processing

• The ETS Data Manager integration is executed on MWF at 11PM
  • Any records that were added since the previous run will be picked up and transferred to the Slate incoming SFTP folder
  • Records are processed by Slate whenever a file is available
  • All files are copied to the incoming/processed folder and kept for 30 days on the SFTP server
  • Files that are processed can be viewed in the Slate sources section

ETS Script Usage

The script to download the test score data from the ETS Data Manager has a number command line options that are applied depending on the use case. The most common scenarios are noted below:

1. Standard automation
   1. Command: node getData.js GRE AUTO
      1. Pulls GRE test score data using the dates from the last successful run of the script.
      2. For example, if the script was last executed on 12/20/2021, the new start date for the pull will be 12/21/2021 and the end date will be the date that the script is executed.
      3. The third item in the command line should be TOEFL if those are the scores that are needed.
2. Manual data pull
   1. Command: node getData.js GRE MANUAL 12/01/2021 12/31/2021
      1. Pulls GRE test score data using the dates that are passed into the script.
      2. For example, the command above will pull all GRE test scores reported between 12/01/2021 and 12/31/2021.
      3. The third item in the command line should be TOEFL if those are the scores that are needed.

Requested Features/Issues

Current Cycle

Release 2022.1.1 [March 2022]

Features to Complete

• Not started

Features to Consider

• Not started

Features Ready for Release

• Not started

Previous Cycles

Release 2022.1.0b [February 2022]

• Changed the settings for the source format from cumulative to incremental to retain the file uploads for troubleshooting feed issues
• Updated the server-side code to set the end date of the data range to the day before the current run
  • This will eliminate the possibility of a gap on the run day since all reports will be historic

Release 2022.1.0a [January 2022]

• Changed the SOAP endpoint to use getScoreLinkDataByReportDate instead of by test date
  • Updated GRE and TOEFL envelope files and getData.js to handle alternate endpoint
  • Pulled all data for report dates between 1/1/2021 to 1/7/2022 and loaded into Slate

Release 2022.1.0 [December 2021] – Initial Release

• ETS Integration Tool
  • Create a server-side application using Node.js that can run on the terminal server to connect to the ETS Data Manager
    • Connects using SOAP
    • Decodes the Base64 encoded data and saves to a local data file
      • File name = TEST_date_string.txt
      • Only write a file if there is data retrieved
    • Must allow for command line operation so it can be easily executed via batch script
      • node getData.js $TEST_TYPE $RUN_TYPE $START_DATE $END_DATE
  • Logging
    • Logs each successful automated execution to a local file for subsequent run decisions
      • File names = last_run_GRE.log or last_run_TOEFL.log
    • Logs each successful automated execution to a persistent log for tracking purposes
      • File name = auto_history.log
    • Logs each successful manual execution to a persistent log for tracking purposes
      • File name = manual_history.log
    • Logs each failed execution to a persistent log for troubleshooting purposes
      • File name = error.log
  • Accepts TYPE=AUTO/MANUAL parameter
    • AUTO-MODE: Uses the last run for the test type to calculate the date range
      • The end date is the current date
      • The start date is the end date from the previous run + 1 day
      • If it has not yet been a day since the previous run, the run is skipped
    • MANUAL-MODE: Retrieves the results based on a date range and test type parameters when executed
• Slate Configuration
  • Add and update the mapping for the GRE Score Data File source format
    • Validate using a test record that the values are being stored in the expected locations in the student record
  • Add and update the mapping for the TOEFL Score Data File source format
    • Validate using a test record that the values are being stored in the expected locations in the student record
  • Configure the source formats to watch for a file and process automatically when present
    • GRE Settings
      
      • Files are dropped in the \incoming\ets folder with the GRE_*.txt mask
    • TOEFL Settings
      
      
      • Files are dropped in the \incoming\ets folder with the TOEFL_*.txt mask
• Integration Server
  • Use Task Scheduler to automate the execution of all processes
    • Processes should execute on the following schedule: MWF after 11PM
      • Execute the process for each test type (GRE and TOEFL)

        • node getData.js _TEST_ AUTO

      • Execute a script to copy the saved data file to the Slate SFTP server
        • C:/Scripts/Powershell_scripts/copy_ETS_to_slate_sftp.ps1