DHS OIG, OIG-18-23, USCIS Has Been Unsuccessful in Automating Naturalization Benefits Delivery (2017)

USCIS

Has Been Unsuccessful in Automating Naturalization Benefits Delivery November 30, 2017 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Washington, DC 20528 / www.oig.dhs.gov November 30, 2017 MEMORANDUM FOR: The Honorable L. Francis Cissna Director United States Citizenship and Immigration Services FROM: SUBJECT: John Roth Inspector General USCIS Has Been Unsuccessful in Automating Naturalization Benefits Delivery Attached for your information is our final report, USCIS Has Been Unsuccessful in Automating Naturalization Benefits Delivery. We incorporated the formal comments from USCIS in the final report. The report contains five recommendations to address USCIS’ training needs, perform a risk-based analysis of all unresolved ELIS technical issues, implement a plan for reducing ELIS technical debt, clearly define agency-wide business goals and objectives, and implement a plan to ensure that ELIS provides USCIS personnel with complete, timely, and accurate data to enable more effective benefits adjudication decisions. Your office concurred with all five recommendations.

Based on information provided in your response to the draft report, we consider recommendations 1, 3, 4, and 5 open and resolved. Recommendation 2 is closed. Consistent with our responsibility under the Inspector General Act, we will provide copies of our report to congressional committees with oversight and appropriation responsibility over the Department of Homeland Security. We will post the report on our website for public dissemination.

Please call me with any questions, or your staff may contact Sondra McCauley, Assistant Inspector General, Information Technology Audits, (202) 254-4100. Attachment DHS OIG HIGHLIGHTS USCIS Has Been Unsuccessful in Automating Naturalization Benefits Delivery  November 30, 2017 What We Found Why We Did This Audit In March and November 2016, we reported on widespread challenges in U.S. Citizenship and Immigration Services’ (USCIS) efforts to automate benefits processing using the Electronic Immigration System (ELIS). We conducted this audit to determine the effectiveness of USCIS’ efforts to automate the N-400 Application for Naturalization. What We Recommend We are making five recommendations to the USCIS Director and Chief Information Officer to improve automation of immigration benefits.

For Further Information: Contact our Office of Public Affairs at (202) 254-4100, or email us at [email protected] As with prior ELIS efforts, USCIS automation of the N-400 Application for Naturalization has not been successful. USCIS deployed this capability in April 2016 to improve processing of approximately 84,000 naturalization applications received each month. However, as before, the ELIS capabilities deployed did not include critical functionality necessary for end-to- end Form N-400 processing. ELIS repeatedly experienced outages and did not always perform as intended.

Also, USCIS did not ensure field personnel were adequately trained to use the new system capabilities prior to deployment. The problems in N-400 automation can be attributed to poor program management practices, which have continued since prior ELIS releases. Given its focus on meeting established system release dates, USCIS did not fully address our prior report recommendations to improve user support, stakeholder engagement, performance measurement, and testing to ensure ELIS met user needs and improved operations. Given the problems encountered in naturalization processing, USCIS has not succeeded in meeting its operational efficiency, customer service, and national security goals.

Instead, ELIS introduced naturalization processing inefficiencies as backlogs increased by more than 60 percent and processing times nearly doubled. Moreover, interviews and ceremonies for at least 10,000 naturalization applicants were canceled, and more than 200 individuals became citizens without proper background checks, posing threats to national security. USCIS recently began efforts to address these challenges; however, only time will tell whether these efforts are effective in delivering needed ELIS capability and realizing intended transformation benefits. Management Response USCIS concurred with our recommendations. www.oig.dhs.gov OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security  Table of Contents  Background ........................................................................................................

1 Results of Audit .................................................................................................. 6 ELIS Problems Hampered Form N-400 Processing .......................................... 6 Long-Term Program Management Deficiencies Not Addressed....................... 19 USCIS Has Not Realized Transformation Program Benefits ...........................

30 Recommendations............................................................................................. 41 Appendixes Appendix A: Objective, Scope, and Methodology ................................... ...47 Appendix B: USCIS Comments to the Draft Report ..................................49 Appendix C: Status of OIG and GAO Prior Recommendations ..................55 Appendix D: Major ELIS Releases 2012 to 2016 ......................................59 Appendix E: USCIS ELIS Interfaces as of April 2016 ...............................60 Appendix F: ELIS Performance Measures and Results as of July 2016 .....61 Appendix G: ELIS Vulnerability Assessment.............................................62 Appendix H: Office of IT Audits Major Contributors to This Report ...........63 Appendix I: Report Distribution ..............................................................64 Abbreviations Customs and Border Protection Chief Information Officer Central Index System CBP CIO CIS CLAIMS 4 Computer Linked Application Information Management System ELIS EPMS FBI FOD GAO IDENT IT NBC OIG OIT OTC PCQS USCIS Electronic Immigration System Enterprise Print Manager Service Federal Bureau of Investigation Field Operations Directorate Government Accountability Office Automated Biometric Identification System information technology National Benefits Center Office of Inspector General Office of Information Technology Office of Transformation Coordination Person Centric Query Service United States Citizenship and Immigration Services www.oig.dhs.gov OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Background United States Citizenship and Immigration Services (USCIS) is the world's largest immigration organization with responsibility for providing accurate and useful immigration information and services to its customers, granting immigration and citizenship benefits, and ensuring the integrity of the immigration system. Each year, USCIS processes millions of applications from foreign nationals seeking to study, work, and visit, reside within, or become citizens of the United States. To carry out this mission, USCIS has 19,000 Federal employees and contractors working at 223 offices worldwide.

USCIS provides services through its headquarters office in Washington, DC; 5 service centers; 29 district offices; 139 application support centers; and 4 regional offices.1 In 2016, USCIS’ budget represented 6 percent of the Department of Homeland Security’s overall budget of $64 billion. Immigration Benefits Delivery USCIS provides approximately 90 different types of immigration benefits to its customers, including citizenship. On an average day, USCIS employees: x process more than 30,000 applications for various immigration benefits; x issue at least 8,000 permanent resident cards; x adjudicate more than 250 refugee applications; and x naturalize nearly 3,000 new citizens. Foreign nationals can apply for U.S. citizenship if they meet eligibility requirements, including being at least 18 years old, demonstrating sufficient knowledge of English and U.S. civics, and meeting continuous permanent residency requirements.

On average, USCIS receives approximately 84,000 naturalization applications per month and naturalizes roughly 700,000 new U.S. citizens each year. After applicants take the oath of allegiance they can apply for U.S. Passports, register to vote, and live in the U.S. as citizens. The following USCIS program offices and directorates have primary responsibility for supporting the naturalization process. x The Field Operations Directorate (FOD) oversees 85 field offices nation- wide that process and adjudicate applications. Within this Directorate, the National Benefits Center (NBC) conducts pre-screening to prepare naturalization application cases for adjudication at field offices.  1USCIS asylum offices and telephone centers, as well as the National Records Center and the National Benefits Center, also provide services to customers. www.oig.dhs.gov 1 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security x The Biometrics Division, within the Immigration Records and Identity Services Directorate, oversees 139 Application Support Centers nation- wide that conduct biometric screening of applicants. x The Fraud Detection and National Security Directorate conducts additional screening if a potential threat is identified during application processing. x The Customer Service and Public Engagement Directorate manages customer inquiries, such as updates on the status of an application, and maintains myUSCIS.gov, the public facing website that interfaces with ELIS.

Using this site, customers can complete and submit naturalization applications and receive status updates online. Naturalization Processing The process to become a U.S. citizen begins when an applicant submits Form N-400, Application for Naturalization, and pays a $725 processing fee.2 An applicant may submit an application by mail to one of three USCIS lockbox locations, or online.3 The NBC conducts several steps to pre-screen each applicant to ensure eligibility requirements are met and, prior to adjudication, identify any derogatory information that could impact a citizenship decision.

USCIS

Immigration Services Officers at field office locations conduct adjudication through in-person interviews and additional background checks. Table 1 lists the key steps in the naturalization process.  2 The $725 fee includes a $640 application fee and an $85 biometric fee. Applicants 75 years or older are not required to pay the biometric fee. Military-based applications are exempt from the $640 fee if applying under section 328 or 329 of the Immigration and Nationality Act.

3 USCIS maintains three lockbox locations in Arizona, Illinois, and Texas, where applicants can mail their completed N-400 applications. www.oig.dhs.gov 2 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Table 1: High-Level Naturalization Processing Steps Phase Naturalization Processing Step PreͲ Processing   1 . USCISsendsanoticetotheapplicanttoconfirmreceiptofthe application. 2 . USCISsendsanoticetothe applicant thatassignsabiometrics appointmentdate,time,andlocation.Duringthebiometrics appointment,theapplicant’sfingerprints,photograph,andsignature arecollected.4 3. USCISApplicationSupportCentercollectstheapplicant’sbiometrics. 4.

TheNBCcompletesmultiplecheckstovalidatetheapplicant’s eligibility,payment,andidentity,andconductsbackgroundchecksto determinewhethertheindividualmayposearisktonationalsecurity orpublicsafety.5 5. Anapplicantreceives anappointmentnoticeforthenaturalization interviewtobeheldinaUSCISfieldoffice. Adjudication   6. USCISImmigrationOfficerconducts anaturalizationinterviewwith theapplicanttoconfirmanswersontheNͲ400applicationand administerEnglishcomprehensionandcivicsexams. 7. Theapplicantmayreceivepreliminaryapprovalfornaturalizationat thistime,orthecasemayrequireadditionaladjudication. Average Timeframe (as of June 2017) 2–3weeks afterfiling 3–5weeks afterfiling 5–8 weeks afterfiling 2–3 months afterfiling 3–5monthsafter filing 4–6monthsafter filing 8.

USCISprovidestheinterviewresultsinwriting, indicating whetherthe applicationisapproved,denied,orpendingfurtherscreening. 9. Theapprovedapplicantreceives awrittennoticeschedulinga date fortheceremonytotaketheOathofAllegiance.6 10. USCISImmigrationServicesAssistantsperformthefinalbackground checksandprintthecertificateofnaturalization. 1–4weeksafter interview 1–4weeksafter interview 11. Theapplicantattends theOathofAllegianceceremony (alsoreferred toasthenaturalizationceremony)andreceivesaCertificateof Naturalization.TheoathisadministeredbyUSCISatanadministrative ceremonyorbyajudgeinajudicialceremony. 5–8monthsafter filing Source: DHS Office of Inspector General (OIG)-generated from USCIS data Historically, USCIS has conducted nearly all of its naturalization processing using paper forms.

This entails significant movement of voluminous paper-based files that are expensive to ship and store, prone to handling errors, and difficult to share both within USCIS and across Federal agencies. Immigration Services  4 USCIS requires all applicants to be fingerprinted for the purpose of conducting criminal background security checks. 5 USCIS requires all applicants to undergo Federal Bureau of Investigation (FBI) criminal background checks before USCIS schedules interviews. 6 Some field offices conduct the applicant interview and oath of allegiance ceremony on the same day. www.oig.dhs.gov 3 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Officers also use multiple USCIS systems to perform background checks, schedule interviews, and render decisions on benefits eligibility.

The Computer Linked Application Information Management System (CLAIMS 4) was the primary system used to process naturalization applications until April 2016 when the agency deployed N-400 processing via the Electronic Information System (ELIS). Automation of the Application for Naturalization USCIS planned to transition naturalization processing from its legacy processing environment to an online, automated environment as part of its long-term Transformation Program. This program began in 2005 as a massive undertaking to modernize processing of all 90 immigration benefit form types. Until 2017, the Transformation Program was managed by the Office of Transformation Coordination (OTC).7 The goals of the Transformation Program are to increase efficiencies in benefits processing, improve customer service, and enhance national security and system integrity.

The objectives of the Transformation Program are to enable — • immigrant applicants to establish accounts with USCIS to file and track the status of their applications, petitions, or requests online; • USCIS adjudicators to have electronic access to applications, petitions, and requests, relevant policies and procedures, and external databases; • USCIS management and personnel to track and allocate workloads; and • USCIS to establish electronic linkages with other agencies, such as the Department of Justice and Department of State, for data sharing and security purposes. The main component of the Transformation Program is ELIS, which is intended to provide a centralized, web-based, case management solution to convert paper- based operations to automated processing of immigration benefits. ELIS also interfaces with a number of subsystems that provide distinct services, such as identity management, scheduling, and printing to facilitate end-to-end processing. ELIS currently processes a total of seven benefit form types and two online services, which represent approximately 25 percent of the agency’s workload.

The OTC deployed processing of the Form N-400, Application for Naturalization, in ELIS on April 13, 2016. A complete listing of all major ELIS releases and their descriptions is provided in appendix D. Prior Government Accountability Office and OIG Work USCIS’ efforts to modernize immigration services have been fraught with false starts, repeated delays, changes in strategy, and scope reductions. Since 2005,  7 The Transformation Program was reorganized under the USCIS Office of Information Technology (OIT) in January 2017. www.oig.dhs.gov 4 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security we have conducted eight audits to examine transformation program initiatives. Summarily, these reports identified numerous deficiencies, such as a lack of performance metrics, ineffective planning, inconsistent stakeholder participation, inadequate system testing, and insufficient user support needed for ELIS to be effective.

We have issued a total of 38 recommendations to date. Further, the Government Accountability Office (GAO) conducted four audits of USCIS transformation activities between 2006 and 2016. GAO reported that USCIS needed to improve transformation planning activities, program and contractor oversight, performance management, communications, and information technology (IT) management practices, among other things. In total, GAO has issued 30 recommendations to address weaknesses in the management and acquisition of the Transformation Program.

The status of all OIG and GAO recommendations is listed in appendix C.  www.oig.dhs.gov 5 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Results of Audit As with prior ELIS efforts, USCIS automation of the N-400 Application for Naturalization has not been successful. USCIS deployed this capability in April 2016 to improve processing of approximately 84,000 naturalization applications received each month. However, as before, the ELIS capabilities deployed did not include critical functionality necessary for end-to-end Form N-400 processing. ELIS repeatedly experienced outages and did not always perform as intended.

Also, USCIS did not ensure field personnel were adequately trained to use the new system capabilities prior to deployment. The problems in N-400 automation can be attributed to poor program management practices, which have continued since prior ELIS releases. Given its focus on meeting established system release dates, USCIS did not fully address our prior report recommendations to improve user support, stakeholder engagement, performance measurement, and testing to ensure ELIS met user needs and improved operations. Given the problems encountered in naturalization processing, USCIS has not succeeded in meeting its operational efficiency, customer service, and national security goals.

Instead, ELIS introduced naturalization processing inefficiencies as backlogs increased by more than 60 percent and processing times nearly doubled. Moreover, interviews and ceremonies for at least 10,000 naturalization applicants were canceled, and more than 200 individuals became citizens without proper background checks, posing threats to national security. USCIS recently began efforts to address these challenges; however, only time will tell whether these efforts are effective in delivering needed ELIS capability and realizing intended transformation benefits. ELIS Problems Hampered Form N-400 Processing Similar to previous automation initiatives, the deployment of ELIS functionality to process naturalization benefits was not successful.

Following its initial release in April 2016, field office personnel reported that ELIS lacked critical capabilities needed for end-to-end Form N-400 processing. Frequent ELIS outages and performance problems further impaired naturalization benefits processing. Also, USCIS personnel were not prepared to use ELIS once it was deployed. ELIS Functionality Does Not Meet User Needs ELIS did not deliver all of the capabilities needed to automate the workflow for processing naturalization applications, which can total approximately 84,000 applications per month.

The electronic capabilities that ELIS needed to be minimally viable in processing N-400s included executing system background www.oig.dhs.gov 6 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security security checks, managing digital content (e.g., uploading and storing documents), printing naturalization certificates, and preparing for and closing out naturalization ceremonies. However, NBC and field office personnel indicated ELIS could not successfully process N-400 cases due to missing or flawed functionality. According to FOD management, the five most significant gaps in ELIS capabilities related to background security checks, contingency plans for ELIS outages, direct scanning to ELIS, certificate printing, and case closeouts. The OIT was working to address these issues throughout our audit.

Failed Background Security Checks ELIS was intended to automate the process for screening applicants for naturalization benefits. USCIS personnel are required to vet each applicant to ensure U.S. citizenship is not granted to those who may be ineligible or may pose national security threats.8 Although ELIS provides the interfaces needed to query other systems to support applicant vetting, the specific checks are actually executed in the external systems. To illustrate, ELIS submits electronic queries to four systems: U.S. Customs and Border Protection’s (CBP) TECS database, the FBI’s National Name Check Program, the FBI’s Integrated Automated Fingerprint Identification System, and the Automated Biometric Identification System (IDENT).9 The systems match the applicant’s biographic data (e.g., name, date of birth) against repositories of personally identifiable information and biometric data (e.g., fingerprints) to verify an applicant’s identity and associated derogatory information, if any. Nevertheless, ELIS allowed cases with inaccurate or incomplete background and security checks to move forward in the vetting process undetected.

For example, USCIS officials stated that between November 2016 and January 2017, ELIS allowed nearly 15,000 applications to advance to the next processing stage without complete FBI name checks. Additionally, more than 225,000 cases moved forward in ELIS without complete TECS checks. Multiple USCIS offices discovered these problems following the ELIS deployment in April 2016; the problems continued to occur throughout our audit fieldwork. Table 2 includes a summary of the background security check failures along with the number of cases and the root causes for each from May 2016 to April 2017.  8The National Background Identity and Security Checks Operating Procedures (NaBISCOP) Handbook establishes standards and requirements for conducting security and background checks. 9 The FBI’s National Name Check Program includes a query against the FBI’s Universal Index, which contains personnel, administrative, applicant, and criminal files compiled for law enforcement purposes. www.oig.dhs.gov 7 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Table 2: Examples of Background Security Check Failures May 2016 to April 2017 Date Problem Identified Root Cause Type of Check TECS check  Phase PreͲ processing through adjudication FBIname check “Justin Time” IDENT andTECS checks PreͲ ceremony (twofinal checksat24 and48hours beforean oath ceremony) 5/2016– 3/2017 11/2016    1/2017 9/2016– 4/2017 x Systemtimeouts betweenthePerson CentricQueryService (PCQS)andELIS x ELISdisplayed inaccurateTECScheck results x ELISincorrectly submittedname variationstotheFBI andELISuserscould notviewwhatwas submitted  x ELISdidnot automaticallyinitiate JustinTimechecks withintherequired timeframes x Checkswerenot completedorfailedto initiate x Resultsofthechecks didnotappearinELIS Total Cases 226,056 PCQSrequesttimedͲ out; ELISincorrectly handledTECSresults ELISincorrectly filteredoutnames duetocodingerrors inNovember2016 andJanuary2017  14,916    441 Proposed Fix PCQSrequestswere extendedto180 secondstodelaytimeͲ outs; AnewPCQSfeature wascreatedtoreturn “nohit”responses Filterswereremoved fromnamechecksand theELISdisplaywas changedsouserscould viewthenames submittedforchecking Unknown Timeframefor conductingcheckswas changedfrom24to48 hoursto1to2 businessdays; Codewaschangedto fixdisplayerrors ELISincorrectly managedapplicants whowerecanceled fromnaturalization ceremoniesand rescheduled;ELIS incorrectlyhandled IDENTresults;timing inELISwascoded incorrectly Source: DHS OIG-generated from USCIS data and documentation As indicated in the table, OTC management identified a number of root causes for these failures.

The FBI name check errors in November 2016 and January 2017 stemmed from mistakes in the underlying ELIS code logic introduced during system development. That is, a developer inadvertently programmed ELIS to filter the spelling of names to exclude certain letter combinations before sending the requests to the FBI. For example, the letter combinations “NO,” “NM,” or “NA” were omitted, meaning that “John Adams” would be submitted as “Joh Dams.” Other root causes pertained to system interface problems that prevented the successful exchange of data between ELIS and other systems. For example, connectivity issues occurred between ELIS and two systems that provide data transfer services—the Enterprise Service Bus and PCQS.

The problem related to the inability of each system to handle large volume requests within a specific timeframe before timing out. Both systems’ interfaces with ELIS have presented significant challenges since 2015. Further, the repeated failures to complete automated security checks were not apparent to NBC or field office personnel at the time of processing, causing them www.oig.dhs.gov 8 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security to believe the checks were completed with no cause for concern. For example, in the event of TECS check failures, ELIS indicated “no hits,” even though connections with the TECS database had timed-out or dropped.

Likewise, ELIS was not designed with the capability to display actual names submitted to the FBI National Name Check Program. Instead, ELIS only provided final results, such as “no records,” to confirm that name checks had been run; the system end-users lacked the means to identify potential concerns or to verify that correct names had been submitted. USCIS personnel indicated that ELIS lacked visibility of records details. This visibility was greatly missed because users had the ability to view such details in CLAIMS 4, the legacy system.

During our audit fieldwork, the OIT was working to increase confidence in ELIS background check capabilities by deploying code fixes to the FBI name check process, TECS checks, and Just-in-Time checks. The OIT was also implementing capability for adjudicators to view the exact names that were run in ELIS. To ensure accurate ELIS background checks in the near term, the agency began conducting quality assurance reviews on all TECS and FBI name check inquiries. This entailed re-running TECS checks outside of ELIS for comparison with results from ELIS.

For the long term, the OIT planned to transition security checks outside of ELIS to a separate USCIS system, Active Tool for Linked Analysis and Screening, previously developed but not widely used for background security checks.10 Lack of Access to Electronic Files when ELIS Was Unavailable USCIS did not have contingency plans in place for field offices to continue working in the event of ELIS outages. Normally, Immigration Services Officers view electronic files of applications and supporting evidence in ELIS while conducting naturalization interviews. However, because the capability to view applicant files offline (i.e., outside of ELIS) had not been established, officers had to have paper files shipped in from the NBC so they could conduct the interviews. The lack of a contingency plan to ensure continuity of operations created additional work for Immigration Services Officers and delayed cases from moving forward.

Given the frequency of ELIS outages, the NBC began routinely shipping paper files to field offices in July 2016 in anticipation of problems, negating the purpose of the automated system. Officers retained the paper files to record the results of the interviews once ELIS came back online. During our audit fieldwork, the OIT was working to establish a contingency plan to ensure that officers could continue naturalization interviews and sustain benefits processing when ELIS was down. This involved instituting a new capability for storing case files and evidence in an electronic document  10Developed by the Fraud Detection and National Security Directorate, the Active Tool for Linked Analysis and Screening is a computer-based platform that screens immigration applications through multiple systems.  www.oig.dhs.gov 9 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security management system outside of ELIS, where officers could view or print files upon demand.

Officers could then proceed with interviews during ELIS outages and update the cases in ELIS when the system was restored. Difficulty Scanning and Uploading Applicants’ Files  Immigration Services Officers could not easily scan and upload to ELIS the documentation (e.g., identification, evidence) they received from applicants during interviews. Instead, officers scanned documents one page at a time, and then saved them to their computer desktops to upload to ELIS. An officer demonstrated that this was a labor-intensive process, requiring several minutes to scan, save, and upload each document.

The inefficiencies in this multi-step process were magnified as applicants often provided numerous documents that needed to be scanned. Also, this process increased the risk of information being lost, uploaded incorrectly (i.e., to the wrong applicant’s file), or not uploaded at all. During our audit fieldwork, the OIT was deploying new capabilities to enable direct scan and upload to ELIS. Specifically, the OIT was developing a one-step feature to enable the officer to scan and upload documents with one click of a button.

According to the OIT, this functionality was successfully deployed and operational in all field offices as of June 2017. Certificate Printing Problems ELIS lacked the capability to print naturalization certificates. Instead, system users relied on an interface with the Enterprise Print Manager Service (EPMS) to send print requests for certificates one at a time to a local printer. This awkward process was unreliable, error-prone, and took longer than required in the legacy CLAIMS 4 system.

The OTC defaulted to using EPMS as a workaround, because it already relied upon the system to produce notices, cards, and booklets for other USCIS services. However, sporadic network connectivity and message delivery failures between ELIS and EPMS disrupted or delayed certificate printing. Additionally, ELIS did not have a batch print request capability to accommodate high-volume printing requirements prior to large naturalization ceremonies. Figure 1 provides an example of a certificate of naturalization. www.oig.dhs.gov 10 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Figure 1: Example Certificate of Naturalization Source: USCIS website Field offices were immediately challenged to accommodate this new, more complicated print configuration using EPMS.

Each workstation had to download an updated version of Internet Explorer and be connected to a local printer. Then, a specific print driver had to be installed and color management had to be configured for each individual workstation to enable printing from EPMS. Field office personnel stated that the work associated with setting up these complex printing configurations was a burden and was frustrating, as it was not previously required with CLAIMS 4. Field office personnel experienced additional frustrations when these settings would reset automatically after the OIT dispatched routine security patches to the workstations.

The time spent waiting on each print request to appear in EPMS after being initiated in ELIS resulted in widespread delays at field offices. ELIS users reported it took an average of 1 to 5 minutes for each print request to be executed in EPMS. In an extreme case, USCIS staff in a Los Angeles Field Office spent more than 2 days printing 87 certificates. The loss of batch printing capability, previously available in CLAIMS 4, further increased the amount of time it took to prepare for naturalization ceremonies.

Such delays in certificate printing were especially problematic for offices such as Newark, NJ, that typically conducted same-day oath of allegiance ceremonies. In such offices, naturalization certificates had to be printed on demand for up to three ceremonies a day, each accommodating hundreds of applicants. Certificates of naturalization are considered secure documents that must be printed on special paper and include a number of mandatory biographic fields and codes. However, using EPMS, dozens of naturalization certificates were printed with missing or incorrect data.

For example, certificates sometimes printed without information, such as photos, or included incorrect name spellings or wrong dates of naturalization. In these cases, the information displayed correctly in ELIS, but the certificate would print with incorrect www.oig.dhs.gov 11 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security information. As such, many certificates were routinely voided and reprinted—an expensive and time-consuming exercise. An Immigration Services Officer in the Newark Field Office explained that although these errors occurred on a weekly basis, they were unpredictable.

During our audit, the OIT was working to simplify the printing process by deploying a capability to print directly from ELIS, thereby eliminating network- dependent interfaces. They were also working to establish a batch printing process in ELIS. Field testing began in March 2017 and the functionality was deployed to all field offices on April 10, 2017. Early testing of this solution was successful, as the FOD reported that local printing in ELIS averaged 2 seconds per certificate.

Inadequate Case Closeouts after Oath of Allegiance Ceremonies  ELIS did not automatically update the USCIS Central Index System (CIS) with final immigrant status for all cases, as required. A direct interface between ELIS and CIS was established to automatically transfer data for each case, including the applicant’s name, certificate number, date of naturalization, and court location. Nevertheless, ELIS did not consistently update CIS due to synchronization failures caused by mismatched data fields. For example, if a change was made to a key data field, such as an applicant name change, data might not transfer to CIS correctly.

Additional problems occurred because court codes were not correctly aligned between ELIS and other systems. During our audit, field office personnel found discrepancies in tens of thousands of cases wherein applicants had been naturalized, but their status had not been updated in CIS. Maintaining accurate records in CIS is critical, as it constitutes the official repository for all immigrant applicants’ status. Also, data from CIS is used by CBP to query the official status for individuals requesting and receiving any type of immigration benefit.

Other CIS update problems were attributed to the specific order of steps to properly close out naturalization ceremonies in ELIS. For example, a case could not be closed in ELIS until the Form N-445, Notice of Naturalization Oath Ceremony, was scanned and uploaded for each naturalized applicant. For large ceremonies, this was a time-consuming process that could require multiple administrators to process a single batch. If one case within a batch was not included, none of the cases in the entire batch could be closed.

At the time of our audit, the OIT was working to ensure that ELIS accurately updated CIS. Specifically, the OIT had developed a daily validation report to identify discrepancies between ELIS and CIS records. The OIT also implemented a work queue to allow ELIS users to view a list of all cases that were not updated in CIS. Although this solution was a manual process, FOD expected that these tools would enable USCIS personnel to quickly remediate any inconsistencies. www.oig.dhs.gov 12 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Additional ELIS Functionality Problems NBC and field office personnel discussed additional functionality that was either missing or not working as intended.

This resulted in additional time and resources to conduct manual workarounds to forward cases from one step to the next. For example, as of January 2017, the only way for the Background Check Unit to refer a case to an adjudicator was to email a notification to the adjudicating officer’s supervisor via Microsoft Outlook. This required working outside of ELIS, which poses a risk of missing the notice. Additionally, we observed personnel at numerous field offices we visited devoting extra time to recheck and validate data across multiple systems because they did not trust ELIS.

This negated the intended efficiencies of automation, and, instead created an environment that was more labor-intensive than legacy processing. NBC personnel estimated that over 70 percent of ELIS cases needed manual intervention to correct problems and prepare cases for interviews. Table 3 provides examples of specific concerns raised by ELIS users. Table 3: ELIS User Feedback Reported in January to February 2017 Functionality Lacking in ELIS Specific Workaround InabilitytoviewRecordsofArrestsand Prosecutions(i.e.,criminalrecords).  Inabilitytoupdatedatafields,suchasapplicant’s countryofcitizenship. Inabilitytoupdatedatafields,suchasapplicant’s namechange. Inabilitytoupdatethesystemwithapplicant’s appointmentarrivaltime,causingthesystemto recordtheapplicantasanoshowandrescheduling theappointment. Reliance onsupervisors todownloadRecordsofArrestsand ProsecutionswhenELISͲuserslackedaccesstotheCustomer ProfileManagementServiceinterfacecontainingthe“RAP sheets.” Editingacertificateonceitwasqueuedupforprinting,rather thaneditingtherelevantfieldsinELIStopopulatethe certificate. “Tricking”thesystembycreatingafalsenameinorderto resetthesystemandallowanupdatetobemade. “Tricking”thesystemintothinkinganadjudicatorwas conductinganotherperson’sinterviewsothattheapplicant actuallybeinginterviewedwasnotrescheduled. Failure of applicant’s signature to print legibly or print at all.

Conducting“artsͲandͲcrafts”tomanuallyfixandhave applicantssignpapernaturalizationcertificates.  Source: USCIS documentation and auditee statements Despite these challenges, the consensus among Immigration Services Officers in many field offices we visited was that ELIS was beneficial “when it worked.” For example, several officers stated ELIS was effective for facilitating interview steps with applicants. ELIS was especially helpful when an applicant neglected to bring required documentation, because the officer could easily generate a reminder notice in ELIS to mail to the applicant. Additionally, officers liked having the ability to directly make notes to a case in the system, which could be helpful to other personnel who worked with the cases downstream. www.oig.dhs.gov 13 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security We previously reported on the Transformation Program’s inability to ensure that each ELIS release delivered the functionality needed to process immigration benefits. The use of manual workarounds constrained resources and has been a prevalent practice since ELIS was released. x x In 2014, we reported that ELIS deployments did not provide the functionality needed to support USCIS’ mission, resulting in a marked slowdown of work processes for adjudicators.11 In 2016, we again reported that ELIS deployments had not included needed functionality, such as case referral, management reporting, or the ability to change a customer’s address.

These deficiencies required manual intervention to move cases forward.12 Poor ELIS Performance Further Constrained Efficiency Although steps have been taken to ensure system security, ELIS performance problems negatively impacted productivity for the NBC and USCIS field offices. ELIS users experienced widespread system outages, poor reliability, slow processing speed, and frequent interface problems. Further, frequent system errors have remained a long-standing issue across ELIS product lines. Up-to-Date System Security Patches Given that ELIS is a web-based application that contains personally identifiable information, information security is critical.

On a positive note, we found that USCIS took steps to ensure ELIS security. Specifically, we conducted technical testing at USCIS headquarters to review the security configuration of the servers that hosted the ELIS website and supporting back-end database. We also examined the deployment of security patches on the underlying operating systems. From this, we concluded that USCIS had implemented an effective patch management program, deploying software updates on a regular basis to reduce vulnerabilities within ELIS.

See appendix G for our vulnerability assessment methodology and analysis. Poor System Availability and Speed ELIS experienced numerous outages and periods of degradations in the months following the initial N-400 deployment in April 2016. Periods of poor system performance occurred on a frequent basis. At least 15 outages and periods of degradations were reported between August and October 2016 alone, totaling 59 hours.

Although certain adjudication steps, such as facilitating interviews, could  11 U.S. Citizenship and Immigration Services Information Technology Management Progress and Challenges, OIG-14-112, July 2014 12 USCIS Automation of Immigration Benefits Processing Remains Ineffective, OIG-16-48, March 2016, Better Safeguards Are Needed in USCIS Green Card Issuance, OIG-17-11, November 2016 www.oig.dhs.gov 14 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security be conducted apart from the ELIS technology, the results could not be uploaded, preventing cases from advancing until the system was back online. Table 4 lists significant system outages and periods of degradations between August and October 2016.13 Table 4: Outages and Degradations (August 2016 to October 2016) Date Type Duration (minutes) 8/9/2016 8/9/2016 8/12/2016 8/31/2016 9/2/2016 9/21/2016 9/22/2016 10/2/2016 10/4/2016 Outage Outage Outage Outage Outage Outage Outage Degradation Degradation 10/13/2016 Degradation 10/14/2016 Degradation 10/17/2016 Outage 10/17/2016 Degradation 60 20 69 311 31 17 17 266 208 108 271 376 31 Description ELIS Subsystem/Interface Subsystem/Interface Other ELIS ELIS Network Subsystem/Interface Subsystem/Interface Subsystem/Interface ELIS Network Subsystem/Interface 10/20/2016 Degradation 1560 Subsystem/Interface 10/21/2016 Outage 174 Network Total    3519minutes(58.7hours) Source: DHS OIG-generated based on analysis of OTC program management review logs Further, ELIS did not meet USCIS’ performance standards for system reliability, which targeted an average of 641 hours of sustained system performance between failures. In contrast, ELIS reliability averaged 453 hours per month in FY 2016. System reliability refers to ELIS’ ability to provide service to end-users and successfully respond to interfaces without interruption.

A September 2016 OIT assessment indicated ELIS had not met the 641 hour system reliability target in 7 of the 12 previous months. The OTC attributed ELIS performance issues primarily to DHS OneNet, the network on which it resided.14 OTC leadership stated that DHS OneNet was not stable over the summer months in 2016, sometimes negatively affecting ELIS  13 Degradation refers to a decrease in connectivity and response speed.  14 DHS created OneNet in 2005 to consolidate component networks into an integrated technology infrastructure. www.oig.dhs.gov 15 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security performance. For example, OTC management asserted that OneNet caused ELIS users to lose access or experience lockups from time to time. OIT management identified ELIS’ dependence on OneNet as a key issue to address going forward.

Field personnel were further encumbered by slow processing speeds when ELIS was accessible. USCIS personnel on the East Coast indicated that processing time was significantly slower in the afternoon when more users were signed onto ELIS. The slow processing speeds had a marked impact on the number of cases that could be worked each day. Our prior audits conveyed similar ELIS performance issues.

For example, in March 2016 we reported that personnel at the NBC and the Texas Service Center struggled with ELIS performance problems that negatively impacted productivity in adjudicating Immigrant Fee and I-90 cases.15 Adjudicators at both service centers intermittently experienced slow processing speeds, as well as frequent system outages. The ability to stabilize ELIS performance given its numerous system interfaces has proven to be a long-standing challenge. ELIS interfaces with nearly 40 distinct internal and external systems that provide specific capabilities or services for end-to-end application processing. As designed, these interfaces should not degrade ELIS performance and should enable continuous processing for the ELIS user.

Yet, significant problems persisted in Form N-400 processing as interfaces either failed or disrupted end-to-end processing. Table 5 includes examples of ELIS interface problems and their impact on N-400 application processing. In 2016, we reported similar problems for previously deployed ELIS capabilities to process other benefit types such as the Form I-90. ELIS Subsystem N-400 Interface Problems Impact Table 5: Examples of N-400 Interface Problems Enterprise Correspondence HandlingOnline NationalAppointment SchedulingSystem •ELISdisplayedthatitwasawaiting generationofcustomernotices,even thoughthenoticeshadalreadybeen sent •TheNationalAppointmentScheduling SystemandELISwereunableto harmonizefieldofficezipcodes •ELISdidnotcommunicatewiththe NationalAppointmentScheduling Systemtomakeappointmentstocollect biometricdatafromapplicants  •CasescouldnotproceeduntilELIS recognizedthatthenoticeshadbeen sent •Applicants weresenttothewrongfield officesforoathofallegianceceremonies •ELISuserscouldnotlocateappointment informationintheappointmenttabor historytab,andappointmentnotices weremissingfromthesystem Source: DHS OIG-generated based on USCIS N-400 incident logs and previous OIG reporting  15 USCIS Automation of Immigration Benefits Processing Remains Ineffective, OIG-16-48, March 2016 www.oig.dhs.gov 16 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Persistent System Processing Errors System glitches, or errors, have persistently disrupted system performance across at least four ELIS product lines.

ELIS users recounted widespread system instances when the system was unable to execute specific functions as intended or did not display data correctly. Table 6 provides examples of such errors. Naturalization Processing Step Interviews BiometricsChecks Adjudication OathofAllegiance Ceremony Table 6: Examples of ELIS Errors in N-400 Processing System Errors While Processing N-400 Cases • • • • • • • • • • • • ImmigrationServices Officerswereunabletochangetheapplicant’scountryof citizenshipinELISduringaninterview Theapplicant’sDateofEntrywouldchangetothewrongdayafterELIStransmittedthe datatotheadjudicator’sconnectedtablettoobtaintheapplicant’ssignature TheConductInterviewTaskremainedpendinginELISeventhoughtheapplicantwas alreadyapprovedforanoathofallegianceceremony ELISwasmissingfingerprintresults,photographs, andsignatures FingerprintIdentificationNumbersdidnotloadintotheReviewFingerprintResultfield inELISasdesigned ImmigrationServicesOfficersreceivednoticesinELISincorrectlystatingthatNͲ400 applicantsdidnotshowupfortheirappointments ELISrandomlychanged thedisplaypagewhenELISusersattemptedtoviewapplicant evidence ELISdisplayedablankscreenwhenusersclickedthebuttoninELIStoviewdocuments TheCaseDecisiondropͲdownboxwasmissingthe"Approved"optionforadjudicators toselectinELIS ELISusersattemptingtoscheduleapplicantsfornaturalizationceremoniesnoticed thattheOathSchedulingtabwasmissingfromthesystem ELISuserswereunabletouploadtheNͲ445NoticeofNaturalizationOathCeremony documenttothesystem ELISwasunabletoassignacertificateofnaturalizationbecauseofmultipleoath appointmentsscheduledforanapplicant Source: DHS OIG-generated based on analysis of USCIS incident logs, 2017 System processing errors were prevalent in other product lines previously deployed in ELIS. In March and November 2016, we reported that similar errors occurred on a regular basis for Form I-90 and USCIS Immigrant Fee for Green Card processing.

Table 7 lists examples of such errors, as reported by ELIS users. www.oig.dhs.gov 17 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Table 7: ELIS System Problems in Previously Deployed Benefit Types I-90 Processing USCIS Immigrant Fee for Green Card Processing • Casesbecamestuckatvariouspoints • Morethan 5,000GreenCardswereissuedwiththeincorrect throughoutprocessingandwereunable tomovetonextstepswithout intervention. namesand/ordatesofbirthduetotechnicalerrors, includingtheimmigrant’sinformationdisplayedonanother familymember’scard. • Carderrorsoccurredwhen“NMN”was enteredforapplicantswithnomiddle name. • Over300GreenCardswereissuedwithincorrectphotosdue toatechnicalglitchthatenabledrandomassociationof photosacrossmultiplefamilymembers. • Cardscouldnotbeproducedfor • AfaultysubͲstatusfieldinELIScausedanumberofduplicate approvedcases. cardstobeproducedfromasinglecasenumber. • Customerdateofbirthdisplayed incorrectly. Source: DHS OIG-generated based on analysis of USCIS incident logs, 2015–2016 Users Were Not Well Equipped to Process N-400 Applications Using ELIS USCIS personnel we interviewed did not have sufficient knowledge to effectively use ELIS or its sub-systems to adjudicate N-400 applications. For example, field office personnel lacked understanding of how to carry out key steps in ELIS to adjudicate and close out cases. Immigration Services Officers we spoke with stated that significant coordination among field office staff, as well as with supervisors and regional contacts, was required to determine how to perform certain steps. To illustrate, personnel in both the New York, NY and Newark, NJ offices recounted the difficulties they encountered in trying to understand why naturalization certificates would not print from ELIS.

They indicated that it took 1 week to reach someone from the OTC to learn how to print the certificates. NBC personnel also struggled to understand the ELIS case access levels that the OTC had put in place to restrict system functionality based on user roles and positions. NBC personnel claimed there was a lack of guidance on the permissions granted at each access level. This resulted in user difficulties understanding when the system was malfunctioning, as opposed to when it was working in accordance with access restrictions.

Lastly, ELIS field office users we met with stated they did not know how to operate peripheral devices, such as connected tablets and scanners, which are used to complete in-person interviews. For example, personnel in the Newark Field Office did not know what the connected tablets were to be used for and faced difficulties learning how to turn them on, change settings, and properly connect them to ELIS. Personnel in the same office did not know how to operate new scanners and had to seek assistance from the OTC. www.oig.dhs.gov 18 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security The lack of training was evident when ELIS cases began to transition to field offices in July 2016. More than half of all support calls to the OTC’s Situation Room in September 2016 were for operational guidance.

Additionally, 126 of the approximately 317 incidents reported to the OTC between August and October 2016 (roughly 40 percent) were categorized as “Training Issue/Question.” ELIS Processing Suspended for New Cases Given the numerous difficulties in processing N-400 cases in ELIS, USCIS leadership made a decision in August 2016 to revert intake of all newly-filed Form N-400s to CLAIMS 416. The decision was prompted by ELIS functionality and performance deficiencies, coupled with a higher than anticipated number of system incidents. The Acting Director of USCIS stated that taking this step allowed the OIT to address the five most significant functional deficiencies identified by the FOD involving background checks, printing, case closeout, scanning, and contingency planning. Agency leadership also anticipated that this pivot back to CLAIMS 4 would provide time for ELIS enhancements to promote field office personnel confidence in the system after it had adversely impacted Form N-400 throughput.

We issued a Management Alert on January 27, 2017, advising USCIS to resolve these Form N-400 processing issues before returning to ELIS to begin working new cases.17 Nearly 250,000 cases were entered into ELIS following its deployment for N-400 processing in April 2016. However, when ELIS was suspended from accepting new cases in August 2016, there were still more than 240,000 N-400 cases in ELIS that needed to be completed. At the conclusion of our fieldwork in April 2017, more than 50 percent (nearly 148,000) were still pending completion in ELIS — a full year after deployment of Form N-400 processing capability. Long-Term Program Management Deficiencies Not Addressed ELIS functionality and performance problems may be attributed to essentially the same challenges that we reported in 2016 as prevalent in previous system releases.

Primarily focused on its ambitious system release schedule, we found the OTC had not yet addressed prior OIG recommendations to improve user support, stakeholder engagement, performance measures, and testing. Further, system design complexities have persisted since the first ELIS release in 2012.  16Although USCIS ceased the processing of new paper-filed applications in ELIS beginning August 29, 2016, it continued to receive and process a small number of e-filed applications in ELIS. 17 Management Alert – U.S. Citizenship and Immigration Services’ Use of the Electronic Immigration System for Naturalization Benefits Processing, OIG-17-26-MA, January 2017 www.oig.dhs.gov 19 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Schedule-Driven Approach Posed Risks The schedule-driven nature of ELIS deployments led OTC management and technicians to focus on speed rather than quality in ELIS system development and implementation efforts. OTC management and IT personnel conceded that ELIS N-400 development activities were geared toward meeting ambitious deployment schedules and frequent deadlines that constrained system developers in delivering functionality quickly to meet agreed-upon release dates.18 Working in this manner favored applying resources to achieve the next ELIS product release as quickly as possible. It discouraged day-to-day oversight of development teams to ensure that functional and technical requirements were delivered, while also adhering to sound IT development and deployment practices.

The tight deadlines also discouraged enhancing the functionality released to process each benefit type (i.e., form) in ELIS after initial deployment. To emphasize, the OIT concluded that the program moved from one product line to the next too quickly, without putting adequate attention into incrementally improving the previous product line. In October 2016, USCIS reported to the DHS Chief Acquisition Officer that this schedule-driven approach was a root cause for the ELIS N-400 processing problems. Despite the schedule-driven approach, the OTC struggled to meet its original target date for the ELIS N-400 release.

According to its 2015 Release Planning Roadmap documentation, the OTC planned a “Go-Live” release date of February 26, 2016, for ELIS N-400 processing capability, followed by continuous deployments of core functionality until May 14, 2016. Nevertheless, the OTC was unable to meet these milestones, encountering delays that exceeded 11 months. Specifically, the OTC pushed the Go-Live date out twice, from February 26, 2016, to March 28, 2016, and ultimately to April 13, 2016, for deployment of the automated N-400 capability. Likewise, targeted completion of continuous deployments of core functionality deployment was delayed eight times until December 20, 2016.

However, as of April 2017, the continuous deployments were still underway 1 year after the initial launch, signifying that the OTC had severely underestimated this effort. USCIS did not recognize the risks associated with this deployment approach. Successful program implementation was contingent upon phased completion and deployment of core ELIS functionality to meet two major stages in automated N- 400 processing: (1) May 2016, when customer applications received in ELIS were initially routed to the NBC for pre-processing, and (2) July 12, 2016, when field offices began conducting interviews for adjudication. However, OTC developers were unable to deliver the necessary functionality to meet these two stages.

As of September 2016, core functionalities, such as ELIS capabilities to reschedule  18The 2015 Acquisition Program Baseline includes a schedule dictating which forms must be implemented by which date.  www.oig.dhs.gov 20 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security interviews and execute case transfers and referrals for supervisory review, were still under development. OTC Program documentation from September 2016 confirmed that these functionality development efforts were still underway, although originally planned for completion by May 2016. Given the challenges associated with N-400 processing capability deployment, the OTC did not meet its overarching milestone of completing the Citizenship Line of Business (including naturalization) by September 2016 as intended. This target date was a pivotal milestone established in the April 2015 Transformation Program Acquisition Baseline.

The failure to meet this milestone resulted in discontinuation of further ELIS product development efforts after the DHS Acquisition Review Board placed the Transformation Program in breach of status in October 2016.19 A July 2016 GAO report forecasted the challenges of the schedule-driven Transformation Program.20 Specifically, GAO anticipated that striving to meet a tight schedule of established dates could increase the risk of proceeding with ELIS deployments before the system was ready. In addition, GAO reported that the program risked future schedule delays in subsequent USCIS ELIS releases, which might result in the program exceeding its established Acquisition Baseline schedule deadline. User Guidance and Support Were Still Lacking The OTC did not have a well-established plan in place to provide training or technical support to ELIS users. This complaint has been prevalent since the first ELIS release in 2012.

Additional Training Was Needed NBC and field users were not prepared to use ELIS for Form N-400 processing once the system was released. The Office of Management and Budget requires that users of Federal IT resources have the skills, knowledge, and training needed to be effective.21 However, the training provided prior to ELIS deployment did not include adequate instructions or hands-on learning opportunities to ensure field officers were prepared to use the system. The USCIS training approach also was not successful, given the added capabilities from ongoing ELIS development efforts concurrent with the training.  19 The DHS Program Accountability and Risk Management Office conducts periodic reviews of acquisition milestones through the DHS Acquisition Review Board. The board, composed of the Under Secretary for Management, the Under Secretary for Science and Technology, and other officials within the Department, makes decisions concerning major investments based on cost, schedule, performance, and risk.

20 USCIS failed to adhere to key Agile development practices identified in Immigration Benefits System: U.S. Citizenship and Immigration Services Can Improve Program Management, GAO-16-467, July 2016. 21 Managing Information as a Strategic Resource, Circular A-130, Section 5(c)(3), July 28, 2016 www.oig.dhs.gov 21 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Specifically, USCIS’ training division organized a train-the-trainer approach to provide in-person or virtual training sessions to designated personnel, such as power users, from the NBC and each field office. This approach was contingent upon personnel learning the system during training sessions so they could, in- turn, train additional users within each field office. However, training sessions were conducted while core system development and deployment efforts were still underway and, as such, the training was not fully effective.

For example, NBC personnel attended the first in-person ELIS training session in April 2016, just before the initial N-400 Go-Live date. Although this training included high-level information on the steps needed, the training did not include specific information on how ELIS would automate distinct processes such as the name harvesting step.22 Without this training, NBC personnel were unclear how ELIS would function or what other USCIS systems needed to be checked. The benefits of training sessions were also limited by a lack of hands-on system usage and reliance on printed materials that did not reflect the true look and feel of ELIS. To illustrate, instructors relied on PowerPoint presentations, participant guides, and screenshots to demonstrate ELIS functionality that was still under development.

NBC and field office personnel we spoke with stated that training materials lacked key information on ELIS functionality, interfaces, and peripheral equipment required for end-to-end application processing. Users’ inability to practice on the system during training greatly inhibited their understanding of ELIS. The users’ first hands-on experience with the ELIS N-400 capability involved adjudicating active cases—a high-risk endeavor. A lack of guidance prior to and during ELIS deployments was a long-standing problem.

USCIS leadership acknowledged that many staff might be uncomfortable switching from paper-based processes to system-based processes in ELIS and that ample training was needed. Further, our July 2014 audit report indicated that additional training was needed to ensure that users were aware of new system features. However, given the limited training provided, users remained ill-equipped to work independently on live cases in ELIS and were overwhelmed by the new functionality introduced with the N-400 release. Insufficient Technical Support Similarly, the OTC deployed ELIS without adequate resources or methods to deliver end-user support.

Having a strategy for delivering technical support was imperative, due to the addition of real-time processing (i.e., conducting interviews with applicants using ELIS) as well as the high number of ELIS users associated with the N-400 release. Specifically, this release was deployed to 85 field offices, the National Benefits Center, and 5 Customer Service centers, totaling 5,000  22Name harvesting entails cross-checking USCIS records to identify additional names or dates of birth that could be pertinent to an applicant. www.oig.dhs.gov 22 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security end-users. By comparison, the two previous major ELIS releases in 2015 were limited to two office locations (a Service Center and the National Benefits Center) and fewer than 2,300 USCIS ELIS users. However, USCIS field offices began N-400 processing in April 2016 without the end-user support needed to ensure success.

OTC personnel we spoke with stated that N-400 support efforts were initially understaffed due to the need for technical staff to begin developing the next ELIS release. This lack of support became apparent to users when ELIS naturalization interviews began on July 12, 2016. In response to the high number of problems that users encountered, the OTC quickly sought to increase ELIS support to the field. In August 2016, the OTC set up a “Situation Room,” for on-demand interview support, staffed with 8 to 10 OTC developers as well as field office personnel.

A conference line was also set up for Immigration Services Officers to call if they experienced difficulty during interviews between 7:30 a.m. and 4 p.m. EST. During the first week of September 2016, field offices required assistance for more than 50 percent of all interviews they conducted. The agency ramped up the number of interviews conducted in ELIS each week from July to October, eventually reaching 5,000 interviews per week in October 2016. The increased volume of interviews in the field, along with other requests for assistance with ELIS-related problems, equated to a proportionate spike in incidents reported, as depicted in figure 2.

The number of incident tickets increased by more than 1,000 percent from August to December 2016, peaking at 1,112 new incidents in the month of December alone. The volume of incidents quickly outpaced the ability of ELIS Situation Room and technical staff to resolve them. Figure 2: ELIS N-400 Incidents Reported by Month 1200 1000 800 600 400 200 0 s t n e d i c n I 0 0 4 - N New Incidents Created Source: DHS OIG-generated based on USCIS OTC incident logs Month In response to the rising number of incidents, the FOD implemented a tiered support system in October 2016 to augment support for its field office personnel. The tiered system was broken into three layers to escalate issues as needed until www.oig.dhs.gov 23 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security resolution.23 Field users described Tier 1 power users and Tier 2 regional support as being both active and very responsive.

However, they indicated that support from the OTC’s Tier 3 Situation Room remained unreliable. Other users said that it was often difficult to reach Tier 3 at the end of the business day, sometimes as early as 3:30 p.m. EST, leaving field offices with no avenue for obtaining technical assistance. In addition, NBC and field office personnel we interviewed struggled to maintain accounts of open incident tickets and cases in their local offices. ELIS users at some locations attempted to keep lists, while others said it was not possible to keep up due to the high volume of open tickets.

Often, the tickets were on hold pending system fixes. However, both NBC and field office personnel stated there was limited communication from the OTC regarding when issues would be addressed. More troublesome, several personnel stated that reported incidents remained unresolved for months, even though the corresponding incident tickets had been closed. Numerous management officials and personnel across FOD took on significant duties to facilitate problem solving and share announcements as ELIS workarounds or fixes were identified.

ELIS users conceded that the communications and support provided by FOD management during ELIS N-400 deployment were essential. Without it, users would have lacked the information necessary to sustain daily operations. For example, FOD authored a news alert in January 2017 to inform ELIS users in all field offices on what to do if naturalization certificates were stuck in the “Print Requested” status and not showing up in EPMS. FOD management employed multiple tools, such as a “FOD Hot News” email blast, to inform field offices of ELIS release updates and offer answers to frequently asked questions.

Supplemental FOD guidance was provided through email, a website, in-person weekly and daily teleconferences, videoconferences, webinars, and videos. For example, FOD management launched a campaign in May 2016 to collect feedback and answer employee questions about transformation efforts, and led a weekly teleconference to share the progress made in addressing ELIS functionality gaps. The lack of technical support during ELIS deployments was a long-standing problem we reported on in the past. Specifically, in March 2016, we concluded the OTC had no process in place for users to request technical assistance with routine or specific system issues.

We also reported that ELIS users had no way of monitoring the status of help desk tickets, nor did they receive communications on ticket resolution. As such, we recommended that the OTC implement a structure to provide adequate support for addressing system issues and assisting end-users following deployment of each ELIS release. By the end of our audit in April 2017, this recommendation had not been resolved by the OTC.  23Tier 1-ELIS “power user,” Tier 2- Designated FOD regional personnel, Tier 3- OTC developers www.oig.dhs.gov 24 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Stakeholder Engagement Has Not Improved As we reported in the past, ELIS stakeholders did not have adequate opportunity to provide input to decisions that could impact day-to-day N-400 benefits processing. Participation in efforts to implement ELIS N-400 processing capabilities was largely confined to a few selected field personnel who bore the burden of all deployment-related decisions.

For example, the OTC hosted calls and meetings on a daily or weekly basis for certain USCIS personnel, such as business owners, to discuss specific functionalities and capabilities under development. However, NBC personnel stated that the need to stagger their time across the frequent, overlapping meetings was highly stressful. For example, an NBC manager recounted that he had to field five to six phone calls per day on the various ELIS product lines or issues. Numerous field personnel told us that the meetings often did not include opportunities for them to have meaningful dialogue with knowledgeable OTC development team members who hosted the calls.

Some meetings also did not have the right officials or offices represented to provide the information needed to support certain decisions. Personnel at the NBC discussed the adverse impact of the frequent meetings, indicating the meetings prevented them from performing their normal day-to-day duties. The time commitment for the few selected personnel was magnified when the OTC was gathering requirements and working on development efforts across more than one ELIS product line simultaneously. This caused field personnel to have to prioritize among meetings and risk missing key decision-making opportunities.

USCIS personnel widely agreed that the ELIS coordination efforts were mismanaged, rushed, and chaotic, leaving personnel overwhelmed and stretched too thin. Our March 2016 audit report similarly indicated that stakeholder involvement and communication regarding ELIS implementation were inadequate. For example, we reported that USCIS’ system implementation approach lacked adequate user input to critical system fixes and enhancements. As such, we recommended that USCIS ensure adequate stakeholder involvement throughout system development and deployment so that each ELIS release would provide needed functionality.

By the end of our audit fieldwork in April 2017, the OTC had not resolved this recommendation.

USCIS

Could Not Measure ELIS Impact The OTC did not have a well-established method for assessing whether ELIS was achieving the outcomes expected from the Transformation Program. Specifically, USCIS management and personnel we interviewed could not conclude whether ELIS had improved efficiency, accuracy, or security in benefits delivery. According to the USCIS Chief Information Officer (CIO), the Transformation www.oig.dhs.gov 25 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Program goals and expected outcomes were neither clear nor focused. Within USCIS, the OTC and OIT collected a number of metrics to monitor ELIS and Transformation Program performance; however, they did not monitor the operational impact or quality of ELIS benefits processing.

In 2015, the OTC established eight metrics as required by the DHS Office of Program Accountability and Risk Management for continuous monitoring of ELIS performance. The OTC reported the results on a monthly basis to the DHS Office of the CIO, including specific targets for ELIS reliability, availability, scalability, and overall program progress. (See appendix F for a complete list of ELIS performance measures and results as of July 2016.) However, we identified no metrics associated with the operational impact or quality of ELIS processing on mission accomplishment. For example, the agency lacked the ability to measure the degree to which ELIS achieved targets for reducing adjudication time (established by USCIS fee rules) and the use of paper in immigration benefits processing.

The Government Performance and Results Act Modernization Act requires that all government agencies track and publish such measures annually.24 Similarly, ELIS users lacked the visibility to determine whether the system was functioning as intended. For example, USCIS personnel stated there was no means to identify when a case was stalled, delayed, or in danger of exceeding established processing times. Likewise, supervisors could not track whether cases were being processed correctly versus when a key automated step may have failed, such as background checks. Instead, cases periodically became stuck or applicant interviews were dropped from scheduling without adjudicator awareness.

NBC and field office personnel conceded that the constant need to check for processing problems, diagnose issues, and locate stuck cases was overwhelming. To illustrate, during our site visit in January 2017, NBC officials told us that they recently discovered that nearly 50,000 cases had become stuck in the A-file request queue in ELIS.25 The inability to measure efficiency and accuracy in ELIS is a deficiency that we discussed in past audit reports. For example, in November 2016, we reported that the OTC lacked sufficient measures to alert personnel when actions were needed to correct data errors or cancel duplicate Green Cards before card printing began.26 Likewise, in March 2016, we reported that USCIS could not assess ELIS’ impact on time and accuracy in Form I-90 and USCIS Immigrant Fee Green Card processing. As such, we recommended that USCIS develop and implement performance metrics to measure operational efficiencies achieved via automation of each benefit type in ELIS.

By the end of our audit fieldwork in  24 Government Performance and Results Act Modernization Act of 2010, Public Law 111–352, 124 Stat. 3866 (2010) 25USCIS is responsible for assigning Alien-numbers (“A”-numbers) to foreign nationals. 26 Better Safeguards Are Needed in USCIS Green Card Issuance, OIG-17-11, November 2016 www.oig.dhs.gov 26 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security April 2017, the OIT was working to establish business objectives that could be used to measure the operational impact and outcomes of working in ELIS. The CIO had identified a total of eight objectives, but these had not yet been implemented. System Testing Has Not Improved The OTC did not adequately test N-400 processing capabilities to detect functionality gaps prior to going live in ELIS.

According to DHS and USCIS guidance, an IT program needs to ensure quality software development through ongoing systems testing and documentation of the results.27 ELIS testing deficiencies included inadequate coverage to address high risk areas, a lack of end-user testing, and poor interface testing. Although we previously reported on ELIS testing issues, the OTC did not fully address previous OIG and GAO recommendations to improve testing to ensure each ELIS release functioned as required prior to deployment. Testing for High-Risk Areas Inadequate testing and oversight of high-risk areas led to numerous problems, such as failure to complete 15,000 background checks, after going live with N- 400 processing. Rigorous quality controls were needed to ensure that software developers adhered to standards to prevent potentially harmful or incorrect code logic from being deployed.

During fieldwork for our November 2016 audit, OTC officials conceded that quality controls and oversight were lacking and that stricter guidelines were needed to require developers to exercise greater caution and control before releasing system changes into production. Yet these problems continued during ELIS N-400 development, as peer inspections were not consistently conducted to review system code before proceeding to production. For example, OTC managers stated that the failure to review code for mistakes led to widespread problems with FBI name checks. Inadequate oversight prevented the OTC from discovering a coding error that eliminated certain letter combinations from applicant names submitted for FBI name checks, resulting in the need to redo more than 15,000 background checks.

In response to the DHS Acting Under Secretary for Management, the Transformation Program and the Office of Program Accountability and Risk Management acknowledged in a remediation plan that inadequate testing in high-risk areas, such as background security checks, was one of the root causes for the schedule breach in September 2016.28 Based on this acknowledgement, the OTC should have coordinated agency-wide to ensure DHS stakeholders used a consistent approach to conducting background checks in ELIS. Further, the  27 Agile Processes and Practices Principles and Guidelines, Version 4.0, December 6, 2013; and, Office of Information Technology Agile Development Policy, Management Instruction CIO-OIT-001, April 10, 2013 28 USCIS ELIS Program Remediation Plan, January 30, 2017 www.oig.dhs.gov 27 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security OTC also should have better coordinated ELIS changes to TECS vetting with CBP, which was in the process of upgrading its system to a web-based, distributed processing environment. Planned OIT remediation steps included quality review of background check results, more explicit test standards, and more consistent peer review of all software code related to background checks. The OIT also anticipated adopting new testing tools to enhance its ability to conduct automated system testing.

End-User Testing End-user testing conducted to support deployment of ELIS N-400 capability had minimal value due to its limited scope. End-user tests were necessary to verify that needed ELIS capabilities, such as conducting interviews, referring cases for review, and rendering case decisions, were delivered as intended. However, following the April 13, 2016 “Go-Live” date, nearly 50 percent (19 of 35) of planned end-user tests from April 19, 2016, to December 22, 2016, were canceled due to time constraints. The extensive cancelations reduced opportunities for users to participate in the testing.

OTC officials said that the tests were carried out by a select number of Immigration Services Officers, power users, and field office personnel, some of whom had also been involved with system requirements or development efforts. The tests completed were helpful in uncovering over 69 software defects. End-user testing effectiveness was also limited due to its pre-scripted approach. As we reported in March 2016, the OTC relied heavily on simple tests of case processing functionality that could execute easily in ELIS (otherwise known as “happy path”).

This approach was not beneficial, as the tests failed to demonstrate issues that could fall outside of the prescribed scenarios tested. End-users widely agreed that this reduced the test to more of a system demonstration than an actual test of functionality. In contrast, they stated, most N-400 cases were complex and required unique consideration. In response to these concerns, OTC officials agreed to work with the FOD to incorporate more “exception path” scripts to ensure that test scenarios mimicked real-world use.  System Interface Testing Although the OTC conducted a number of live interface tests before deploying ELIS N-400 processing capability in April 2016, these tests were not successfully completed.

USCIS’ Transformation Program Test and Evaluation Master Plan states that end-to-end testing should be performed to verify that internal and external system interfaces successfully communicate and exchange data in an effective manner and that system functionality meets business requirements. In line with this requirement, the OTC conducted at least six live interface tests, from March 17 to April 6, 2016, in the pre-production test environment; however, these tests were not adequate to validate system functionality would www.oig.dhs.gov 28 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security work as intended upon deployment. Specifically, in April 2016, the OTC reported that live interface testing was insufficient due to the configuration of the pre- production environment. In other words, the capabilities tested failed to expose system interface problems or other unexpected conditions that could occur in the production environment.

Recommendations to Improve Testing Were Not Addressed These ELIS testing deficiencies were not new. In March 2016, we reported similar deficiencies for previously-released ELIS product lines, disclosing that system tests were only performed on specific functions rather than the end-to-end process. We concluded that testing was inadequate to ensure that the automated system provided the functionality needed to be more efficient than the existing paper process. We recommended that USCIS establish a plan for end-user involvement in end-to-end testing to ensure each ELIS release functioned as required prior to deployment.

As of April 2017 completion of our audit fieldwork, this recommendation had not been resolved. Likewise, GAO concluded in July 2016 that testing limitations contributed to ELIS performance issues. Specifically, GAO reported that USCIS was not consistently performing unit and integration testing, functional acceptance tests, and code inspection consistent with guidance and leading practices. GAO further reported that test plans, cases, and results were not fully developed for interoperability and end-user testing.

Until USCIS addresses these long-standing deficiencies, it will be unable to ensure that each future ELIS release is performing as intended, and is not increasing risks to the agency. Architectural Complexity Was Not Addressed ELIS has a complex architecture with dozens of interfaces, processes, and functions, all of which pose challenges for sustaining its existing processing workload and expanding to accommodate additional benefits delivery. Specifically, the system was designed to distribute processing functions across more than 50 interfaces to support the transmission of data and services. Direct connections have been established between ELIS and at least 19 internal and 35 external systems.

(See appendix E for the ELIS interface diagram for N- 400/Release 7.1.) USCIS management and personnel we interviewed described ELIS as a “monolithic” system that is overly reliant on its many subsystems. The USCIS CIO acknowledged that many of the ELIS subsystems are too interdependent, causing unnecessary interactions that increase potential points of failure. The complex architecture also hindered testing, by requiring more time to coordinate and conduct testing of each interface to determine how well it would work in field office locations. Testing of integrated systems was generally limited www.oig.dhs.gov 29 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security to stagnant test environments, which did not generate meaningful feedback on how well capabilities would perform once deployed to multiple field offices.

For example, the OTC released EPMS print capabilities that functioned successfully in the test environment at headquarters, but they failed once deployed nation- wide due to dynamic and unexpected conditions in the production environments. Architectural complexity has been a long-standing issue with ELIS, dating back to the original ELIS deployment in 2012. In March 2012, as part of their oversight activities, the Office of Management and Budget and the DHS Office of the CIO raised significant concerns regarding ELIS’ architectural complexity, given that the first iteration of the system entailed integration with 29 different commercial-off-the-shelf software products. USCIS agreed that the ELIS system architecture was not scalable, sustainable, or flexible and did not meet departmental requirements.

To address this problem, the Transformation Program began an effort in September 2012 to design and build a more modern ELIS architecture, initially known as ELIS 2 but now just called ELIS. Despite these efforts, ELIS’ design remains difficult to expand or maintain and developers have difficulties integrating new product lines, such as N-400 processing, into ELIS. The USCIS CIO acknowledged that the Transformation Program has not taken sufficient time to address the underlying ELIS deficiencies and architectural complexities that led to a high accumulation of “technical debt.” Technical debt encompasses a variety of system development shortcomings, such as data inconsistencies and poor code quality, which can cause unpredictable performance issues and failures during processing. According to the USCIS CIO, not enough technical team capacity was devoted to remediating ELIS technical debt, reducing system interdependencies, and improving software code maintainability.

We also found that past efforts to reduce ELIS technical debt were obstructed by emerging schedule-driven priorities. Specifically, the OTC launched a “Time to Fix” initiative in July 2016 to reduce technical debt levels. Although the initiative began with dedicated resources, it became overwhelmed by new priorities within the first few weeks.

USCIS

Has Not Realized Transformation Program Benefits Given the widespread problems encountered in developing and deploying ELIS, USCIS has not yet fully realized the benefits expected from transforming its paper-based operations into an automated system. Despite its goals to increase efficiency, customer service, and security, evolving use of ELIS has resulted in additional workloads, more processing time, and greater security risks as new product lines are added and deployed. ELIS performance has not been the only difficulty encountered. As the Transformation Program has evolved to meet new priorities, ELIS costs have exponentially increased and scheduled system milestones have repeatedly been breached.

USCIS recently undertook efforts to www.oig.dhs.gov 30 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security address these challenges; however, only time will tell whether these efforts will ultimately be effective in delivering needed ELIS capability and realizing intended benefits of transformation. Inability to Meet Transformation Goals USCIS has not fully realized its automated processing goals through deployment of ELIS. In 2007, USCIS established three primary goals for successful transition to ELIS, an account-based system capable of processing and managing all customer applications electronically. For each benefit type automated in ELIS, USCIS aimed to — x ensure operational efficiency to reduce immigration benefit backlogs and x achieve cost effectiveness and consistent results; improve customer service through timely and accurate adjudication of benefits; and x enhance national security by ensuring that ineligible individuals are not granted immigration or citizenship benefits.

In March 2016, we reported that USCIS had not achieved these goals through prior efforts to automate immigration benefit processing (e.g., I-90 and USCIS Immigrant Fee). This was also the case for deployment of automated processing of naturalization benefits, which was still unfolding throughout our audit. Rather, ELIS has increased workloads and costs and added to the processing backlog. ELIS has also adversely affected processing timeliness and customer service, thereby increasing the national security risk of granting benefits to individuals who were not properly vetted.

Operational Efficiency Not Achieved in Form N-400 Processing USCIS has not met its goal of ensuring operational efficiency and reducing backlogs in Form N-400 processing through the use of ELIS. Rather, we found that ELIS had slowed N-400 processing, resulting in the largest case backlog in more than 5 years. To illustrate, USCIS issued approximately 8,000 naturalization certificates through ELIS per month from April 2016 to April 2017. This volume was equivalent to the number of naturalization certificates issued through the legacy CLAIMS 4 in just 4 days.

Instead of decreasing the complexities of paper-based adjudication that accompanied use of CLAIMS 4, ELIS introduced manual and labor-intensive steps to complete case processing. For example, the laborious process to transmit certificate print requests from ELIS to EPMS entailed several hours of printing certificates, which previously took only seconds. At the New York Field Office, Immigration Services Assistants stated that it took an estimated 45–50 minutes to print a total of 150 certificates, compared to only 30 minutes in www.oig.dhs.gov 31 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security CLAIMS 4. A lower rate of productivity can be expected with any new system implementation, allowing time for personnel to adapt to using the system and becoming familiar with new business processes.

However, users attributed ELIS’ operational inefficiencies to complex workflows that required them to navigate across multiple screens and interface with multiple subsystems to perform specific tasks. Likewise, personnel at the NBC said the time they required to prepare each case for an interview increased by 70 percent, from 63 days in CLAIMS 4 to 108 days in ELIS as of February 2017. Consequently, the number of cases prepared for interview each month dropped by more than 50 percent following the transition to ELIS in July 2016, as depicted in figure 3. According to our analysis of USCIS documentation, average processing time from application submission to applicant interview increased 100 percent, from 105 to 210 days.

Figure 3: Naturalization Interviews Scheduled Each Month, From May to November 2016 29 Source: DHS OIG analysis of monthly metrics reported by the NBC The adverse impact of ELIS N-400 processing deployment was evident in recent agency-wide performance results, which indicated a precipitous drop in naturalization approvals and a spike in the number of pending cases. In April 2017, USCIS reported a backlog of 632,937 cases from the first quarter of FY 2017, an increase of roughly 60 percent from the first quarter of FY 2016.30 It should be noted that there was a 28 percent increase of N-400 receipts during this same time. By contrast, when using CLAIMS 4, the backlog only increased 2 percent from the first quarter of FY 2015 to the first quarter of FY 2016, as depicted in figure 4.  29Both CLAIMS 4 and ELIS were used to conduct Naturalization interviews during this time. 30 First quarter, FY 2017 naturalization data includes non-military applications pending as of December 31, 2016, and processed using both ELIS and the legacy CLAIMS 4. www.oig.dhs.gov 32 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Figure 4: Increases in the Form N-400 Backlog USCISNͲ400Backlog s n o i t a c i l  p p A 0 0 4 Ͳ N 700,000 600,000 500,000 400,000 300,000 200,000 100,000 Ͳ FY2015Q1 (CLAIMS4) FY2016Q1 (CLAIMS4) FY2017Q1 (ELIS&CLAIMS4) Source: DHS OIG analysis of USCIS Quarterly Performance Metrics ELIS also did not help USCIS achieve cost-effective and consistent results. Work to complete processing of the roughly 243,000 N-400 cases accepted into ELIS between April and August 2016 became more laborious.

NBC and field office personnel had to perform duplicate work using legacy systems and processes because Form N-400 automation in ELIS was deployed without all required capabilities. The personnel also did not yet trust the system to produce consistent and accurate results. Further, widespread background security check failures resulted in duplicate efforts as personnel re-ran the checks to ensure that the results were accurate before citizenship was granted. Following are examples of additional work requirements and costs that could have been avoided if background security checks in ELIS had functioned as designed. x Following the ELIS deployment, the NBC adopted a protocol of continuous quality assurance to validate TECS background checks on a daily basis for 4 months, starting in June 2016.

This effort, conducted by four Background Check Unit personnel, required re-running outside of ELIS all TECS checks, which occur during the Form N-400 pre-processing phase. NBC personnel had to compare each ELIS receipt number against the primary name and date of birth resulting from the TECS check query to ensure ELIS was correct. Results were manually maintained in spreadsheets and shared with OIT management to help identify ongoing discrepancies and keep a record of ELIS errors. x NBC devoted a significant amount of time to re-run all of the TECS checks previously submitted as of February 2017, totaling nearly 230,000 cases. www.oig.dhs.gov 33 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security x Field office personnel devoted extra time to respond to the FBI name check failures in November 2016 and January 2017. All field offices were instructed to place active cases on hold to prevent applicants from proceeding to approval or oath of allegiance ceremonies without proper background checks.

Personnel nation-wide had to spend time locating case files, reassigning case files to different work queues, and confirming that case “hold” status was reported back to USCIS headquarters. In addition, all cases that were scheduled for oath of allegiance ceremonies had to be removed from the schedule, in the event that FBI name check results could not be completed in time. x The OTC resubmitted FBI queries for all 15,357 cases affected by ELIS name check failures in January 2017. This totaled approximately $274,000 in unexpected costs to the agency.31 As an additional example, the agency faced unexpected costs in reverting to CLAIMS 4 processing in August 2016 when ELIS was suspended from accepting new cases due to myriad performance problems. The ELIS suspension required that the agency ship hard-copy case files from the NBC to the field offices to support applicant interviews, which cost approximately $400,000 per quarter, or $1.6 million per year.

This defied the key purpose for transitioning to ELIS, which was to reduce the need to ship and store hard-copy files. Naturalization Customer Service Not Improved USCIS did not meet its goals to improve customer service through timely and accurate adjudication of naturalization benefits. Conversely, Form N-400 applicants faced longer than usual delays in getting their cases completed. Prior to the ELIS deployment, the agency had consistently exceeded its published goal of 5 months or less for processing a naturalization application from system entry to an oath of allegiance ceremony.32 Whereas naturalization case completion times averaged 5 months for all USCIS field offices in FY 2015 and FY 2016, as of February 2017, the average processing time had increased to about 8 months.

Also, more than 10,000 applicants faced significant delays from canceled interviews and ceremonies that were attributable, in part, to the cumbersome transition to ELIS. Although a learning curve can be expected with any new system implementation, deficiencies in ELIS training resulted in a more difficult transition than necessary. Because personnel in field offices were unprepared to use ELIS when cases started arriving for adjudication in September 2016, users relied on trial and error to learn the system, which sometimes led to interviews and ceremonies being improperly scheduled or canceled. To illustrate,  31 USCIS claims the average cost incurred for the FBI to process a name check is $17.85 per case.

32 The official processing time goal is documented in the Adjustment of the Immigration and Naturalization Benefit Application and Petition Fee Schedule, 72 FR 29851, May 2007. www.oig.dhs.gov 34 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security approximately 4,200 naturalization interviews were canceled during 2016. Also, more than 6,100 applicants were affected by canceled oath of allegiance ceremonies during the same time. Interviews and ceremony cancellations may also be attributed to other factors, such as an applicant’s change of address, scheduling conflicts, or additional background vetting. Oath of allegiance ceremony cancellations also resulted from the difficulties in printing naturalization certificates.

For example, over the course of 2 days in January 2017, we witnessed the cancellation of 6 ceremonies at the Newark, NJ field office due to an inability to print certificates of naturalization from EPMS. This affected more than 300 approved applicants who would have otherwise been naturalized on the same day of their interviews. Of note, applicants could also be dropped from scheduled interviews or ceremonies due to scheduling conflicts or national security concerns that required additional applicant vetting. More disconcerting is the risk that, due to ELIS errors, an applicant may not be able to prove U.S. citizenship after being naturalized.

As we previously stated, ELIS does not consistently update applicant immigration status in CIS, which CBP officers use to screen arrivals at U.S. ports of entry. If the officers cannot verify status information, a recently naturalized individual could be refused entry into the country. As a recourse, CBP officers would need to conduct additional research, cross-check other systems, or call DHS personnel with ELIS access to validate the individual’s U.S. citizenship. We previously reported in March 2016 that customers were detained for up to a day while CBP officers endeavored to obtain verification of their permanent residence status, as CBP officers typically lacked access to ELIS.

Increased National Security Risks through Improper Vetting of Citizenship Applicants The Transformation Program did not meet its goal of enhancing national security by ensuring that only eligible individuals are granted immigration or citizenship benefits. As we reported in the past, the move to electronic processing in ELIS resulted in greater national security risks as USCIS could unknowingly naturalize applicants with incomplete or inaccurate background security checks. Our audit of Form N-400 automation confirmed that recurring background security check failures in ELIS caused applicants to move forward in the naturalization process when they should have received additional vetting. For example, in May 2016, the NBC identified at least 13 cases that should have warranted additional screening by the Background Check Unit; however, the cases were cleared based on faulty “no hit” results in ELIS.

Of these cases, at least three contained derogatory information with potential national security concerns, such as whether an individual was a suspected or known terrorist. Fortunately, the NBC later cleared these three individuals through additional vetting prior to them being approved for naturalization. www.oig.dhs.gov 35 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security In total, at least 15,000 cases moved forward to field offices for adjudication without proper FBI name checks. Of these, over 200 individuals were naturalized without adequate vetting. The FOD alerted USCIS field offices and provided instructions on identifying and putting these cases on hold while the NBC re-ran the name checks.

After all name checks were completed, USCIS determined that the citizenship approvals were appropriate in each case. The repeated security check failures weakened USCIS Immigration Services Officers’ confidence in their ability to properly adjudicate cases using ELIS automation. Specifically, adjudicators voiced concern about whether the correct name and date of birth were run for each applicant and whether the results were accurately displayed in ELIS. Officers at every location we visited expressed doubts, stating they were not inclined to make adjudication decisions without first checking applicant information in multiple sources, including legacy systems, apart from ELIS.

USCIS officials also expressed concern that the continued problems plaguing security check functionality in ELIS had resulted in a significant loss of trust in the system by field office users. ELIS has been fraught with security risks since its initial deployment in May 2012. For instance, in March 2016, we reported that USCIS had sent potentially hundreds of Green Cards to wrong addresses due to an ELIS limitation that prevented USCIS personnel from updating customer addresses. Additionally, in November 2016, we reported that ELIS design and functionality problems resulted in USCIS receiving more than 200,000 reports from approved applicants about missing Green Cards.

The possibility that these missing Green Cards may have fallen into the wrong hands raised significant security concerns. Further, Green Cards issued with incorrect information or photos can have severe consequences. For example, unaccounted for Green Cards can be used by terrorists, criminals, and undocumented aliens to remain in the United States and access immigrant benefits illegally. Drivers’ licenses, firearms, and concealed handgun licenses may also be obtained by cardholders in certain states without restrictions.

Recurring Changes to the Transformation Approach Overall, the agency has changed the Transformation Program acquisition and deployment strategy three times since its initial Acquisition Program Plan in 2007. With its initial transformation approach, USCIS expected to deploy ELIS for all benefit types by 2013. However, USCIS spent more than $500 million between FY 2008 and FY 2012 to deploy one product line in ELIS.33 USCIS revised its Acquisition Program Baseline twice since 2008, but has been unable to execute any of these plans. These challenges have resulted in recurring schedule delays, totaling more than 4 years, and increased cost estimates of  33 Form I-539, Application to Extend/Change Nonimmigrant Status, deployed in 2012 www.oig.dhs.gov 36 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security more than 400 percent.

Table 8 summarizes each Transformation Program Baseline and the outcomes.34 Table 8: USCIS Transformation Program Plans 2009–2016 Dates Milestone Status Outcomes Reason 1.OriginalAcquisitionProgramPlan,2008 USCISplannedto 2008 Program developand Baseline implementELISfor alllinesofbusiness byFY2013.  Initialmilestones weresettodeploy citizenshipͲrelated benefittypesby September2009. Programestimated costat$536million. implementELISfor alllinesofbusiness byJune2014.  Estimatedtotalcost at$2.1billion. implementELISfor alllinesofbusiness byMarch2019. Programestimated totalcostat$3.1 billion(anincreaseof approximately$1 billion).38 2.RevisedAcquisitionProgramBaseline,201136 The2011 USCISplannedto Program developand Baseline Not Met 3.RevisedAcquisitionProgramBaseline,2015 37 The2015 USCISplannedto Program developand Baseline: ELIS2 Not Met Not Met Ͳ Extendedthetimelineforthe Ͳ USCISdidnothaveadequate resources Ͳ USCISidentifiedmultiple problemswiththesystem architectureandqualityofthe codewhichwouldimpede futuredevelopmentwork initialdeploymentofELISfrom 2009to2011,thedatefor ReleaseAwaspushedbackfour times.35 Ͳ Reducedthescopeofthe deploymenttoonebenefittype. Ͳ Finalreleasedateforthe IͲ539wasMay2012. Ͳ USCISspentmorethan$500 millionbetweenFY2008andFY 2012developing‘ELIS1.’ Ͳ Threeformsweredeployedin ‘ELIS1’in2012and2013,but weredecommissionedby2015. Ͳ Ultimately,theschedulewas delayedby4years,causinga programbreachinJanuary2012. Ͳ ‘ELIS 1’wasshutdowninApril 2016.TheOTCdeterminedthat itcouldnotreuse‘ELIS1’,even afterhavinginvested$500 millioninitsdevelopment. Ͳ TransformationProgramwas placedinaBreachstatusin October2016. Ͳ ‘ELIS2’wasturnedofffornew NͲ400casesinAugust2016. Ͳ ‘ELIS1’systemarchitecturewas overlycomplex. Ͳ Acquisitionstrategyreliedona singlecontractor. Ͳ Developmentmethodologydid notallowthegovernmentto foreseeproblemsearlyenough totakecorrectiveactions. Ͳ Lackofgoalclarity. Ͳ Lackoffollowthroughonthe MinimallyViableProductonce launched. Ͳ Inadequatetestingforhighrisk areas. Ͳ Highaccumulationoftechnical debt. Source: DHS OIG-generated based on analysis of USCIS Transformation Program planning documents  34 USCIS is required to adhere to DHS policies and guidance for major investments, which includes the development of an Acquisition Program Baseline. 35 Release A deployment was pushed to April 2011, then August 2011, then December 2011, then deployed in May 2012. 36 USCIS Transformation Program Acquisition Program Baseline Version 1.3, approved June 6, 2011 37 USCIS Transformation Program Acquisition Program Baseline, Version 2.3, approved April 1, 2015 38 The program baseline approved in 2011 was based on costs from FYs 2006 to 2022. The program baseline approved in April 2015 was based on costs from FYs 2006 to 2033. www.oig.dhs.gov 37 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Moreover, difficulties in deploying ELIS N-400 processing capability caused the agency to fail to meet the acquisition baseline it had established in 2015 to release the Citizenship Line of Business in ELIS.

USCIS’ inability to meet this September 30, 2016 milestone represented the third failed attempt to deploy the system in accordance with approved plans. As a result and as previously discussed, the DHS Acquisition Review Board placed the Transformation Program in breach of status in October 2016. The Deputy Under Secretary for Management directed USCIS to stop system planning and development and focus instead on improving and stabilizing the ELIS functionality already deployed. Additionally, the Deputy Under Secretary for Management directed USCIS to develop and provide to the DHS Acquisition Review Board a proposal for reorganizing the Transformation Program and a remediation plan for addressing ELIS deficiencies going forward.

Recent Actions to Improve the Transformation Program In the wake of the October 2016 program breach, USCIS has undertaken considerable effort to get the Transformation Program back on track. As directed, USCIS halted development and deployment of any additional benefit types in ELIS. The agency turned its focus to addressing deficiencies in the seven benefit types and two online services that were already deployed in ELIS:39 1.

USCIS

Immigrant Fee payment* 2. Application to Replace Permanent Resident Card (Form I-90) 3. Application for Employment Authorization (Form I-765) 4. Consideration of Deferred Action for Childhood Arrivals (Form I-821D) 5.

Application for Naturalization (Form N-400) 6. Application for Temporary Protected Status (Form I-821) 7. Request for a Hearing on a Decision in Naturalization Proceedings Under Section 336 (Form N-336) 8. Application for Replacement Naturalization/Citizenship Document (Form N- 565) 9.

Application for a Travel Document (Form I-131A)* On February 1, 2017, USCIS submitted a Remediation Plan to the Deputy Under Secretary for Management that discussed root causes for the program breach and proposed corrective actions. The plan, approved on February 14, 2017, would be accompanied by a series of updated program documents (e.g., test plans, acquisition plan, and risk register) providing a framework for development and deployment of any future ELIS capabilities.  39 The form types noted with an asterisk indicate an online service. www.oig.dhs.gov 38 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security In January 2017, the USCIS Director developed a new governance structure by placing the Transformation Program within OIT and under the auspices of the USCIS CIO. The reorganization was intended to improve ELIS’ technical foundation through enhanced integration with external systems. The Transformation Program could leverage existing OIT technical skills, functions, and IT development practices to streamline teamwork and reduce redundant efforts.

The USCIS CIO staffed the organization with a new Program Manager, Chief Technology Officer, and Lead Systems Engineer. Additionally, technical teams were assigned a series of initiatives to increase ELIS’ availability and performance, as well as the quality and consistency of system testing. For example, technical teams adopted several new testing tools to address technical debt and improve code maintainability. The USCIS CIO took charge of the Transformation Program in January 2017.

A first step involved updating the strategic direction and scope of the program to optimize the benefits of existing ELIS processing capabilities before undertaking new ones. The CIO then outlined actions to address the four root causes of the 2016 program breach, previously identified in the Remediation Plan. Table 9 identifies these root causes and actions. Table 9: OIT Actions to Address Root Causes of Program Breach Root Cause ThescheduleͲdrivenapproachto launchnewformsdidnotensurethat productlinesweretransformational andbusinessgoalsweremet. TheTransformationProgramdidnot adequatelyimproveproductlinesafter theylaunched. OIT Actions Identified19specificgoalstoensurethatdesiredbusinessoutcomes areobtained.ThesegoalsincludedimprovingtheFormNͲ400 Application,decreasingELIS’dependenceontheDHSnetwork;and improvinginterfaceperformance. Planned tomeasuresuccessbytheaccomplishmentofgoalsandnot bynumberoflaunchedforms.Productownerswouldalsobetasked withensuringthatthesegoalsarecompleted. Backgroundsecuritycheckswerenot testedthoroughly,andTECSvetting wasnotcoordinatedwellwithCBP. Technicalteamsdidnotadequately remediatetechnicaldebtorimprove codemaintainability. Setstandardsforcodetestingandincreased coordinationwithCBP. USCISwouldalsoinvestigateissueswithbackgroundchecksandcreate newstandards. Allocated 20percentcapacitytoreducetechnicaldebtandimplement a3–6monthincreaseintechnicaldebtremediation.OITwouldalso usemicroservices40toincreasetestingandreliability. Source: OIG-generated based on analysis of USCIS documentation, 2017 Further, the CIO identified a new management approach for ongoing and future ELIS development.

Key elements included improved program oversight, user feedback, and technical innovation. For example, a new set of eight business objectives would be used to measure the operational impacts of ELIS, such as adjudication time, reduction of paper, and security. A Business Oversight Board  40Microservices involve continuous delivery of single-purpose services to create a more agile and scalable architecture. www.oig.dhs.gov 39 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security would meet periodically to coordinate transformation activities across USCIS. A new framework for obtaining stakeholder feedback entailed aligning portfolio managers with each USCIS Directorate to provide a direct relationship between the OIT and users.

The OIT expected this framework to improve communications and management of day-to-day ELIS development and testing activities. Also, the CIO anticipated this approach would help ensure that deployed ELIS capabilities would meet the business needs of each USCIS Directorate. Finally, the CIO proposed that the agency would retain the existing Transformation schedule, set for completion by the end of the second quarter of FY 2019. The agency also would not change the Transformation budget, estimated at $3.1 billion through 2033.41 However, the agency would significantly decrease the program scope by reducing the number of automated immigration benefit types from 90 to roughly 15.

This meant continued processing with the immigration benefit types already automated in ELIS, while adding a few other forms such as the I-130, Petition for Alien Relative; I-140, Immigrant Petition for Alien Worker; and I-485, Application to Register Permanent Residence or Adjust Status. The CIO claimed that automating just 15 benefit types would equate to roughly 70 percent of the agency’s total workload. In March 2017 the CIO briefed the DHS Acquisition Review Board on the root causes for the breach and the remediation steps taken to prevent reoccurrence of such problems in the future. As outcomes of this briefing, ELIS development would remain paused and key plans would be updated.

The board requested that the Transformation Program complete the following 12 additional actions before the program could exit breach status and resume automating other benefit types. 1. Conduct Quarterly Program Reviews 2. Update the Concept of Operations 3.

Update the Operational Requirements Document 4. Create a Business Owner Board Charter 5. Create an Executive Steering Committee Charter 6. Develop a Governance Structure Document 7.

Develop a Release Roadmap 8. Update the Acquisition Plan 9. Support an Independent Cost Assessment 10. Update the Program Life Cycle Cost Estimate 11.

Update the Test and Evaluation Master Plan 12. Update the Acquisition Program Baseline document USCIS’ corrective actions were still in progress at the completion of our audit fieldwork in April 2017; as such, the outcomes remain to be seen. A number of the Transformation Program’s original objectives may not be achieved at all given  41The latest cost information available, as of March 17, 2017, included a potential decrease in operations and maintenance spending, which would reduce the $3.1 billion estimate by $0.6 billion. www.oig.dhs.gov 40 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security the reduced scope of this revised approach. For example, USCIS will continue to rely upon CLAIMS and manual processes along with ELIS to deliver immigration benefits; the plan envisioned to decommission legacy systems upon full deployment of ELIS will be lost.

Correspondingly, the operational cost savings anticipated through full ELIS automation will not be fully realized. However, given the system performance deficiencies, user frustrations, time and cost expended, and risks posed by the approach taken heretofore, this program re- scoping and redirection could still prove to be just what is needed to get ELIS on track. Accomplishing some sustained measure of efficient automated benefits processing could ultimately make these initiatives worthwhile. Recommendations  We recommend that the USCIS Director:  Recommendation 1: Assess and address the ELIS training needs of USCIS field offices and service centers to ensure that ELIS users receive sufficient, hands-on training prior to each release.

Recommendation 2: Perform a risk-based analysis of all unresolved ELIS technical issues to ensure that program resources are dedicated to addressing the highest risks impacting the integrity, operational efficiency, and delivery of customer service in immigration benefits processing. Recommendation 3: Develop and implement a plan for reducing ELIS technical debt as a means of improving current benefits processing automation and long- term architectural stability. Recommendation 4: Clearly define agency-wide business goals and objectives for each benefit type automated in ELIS and monitor program performance against accomplishment of these objectives, rather than by release schedule. Recommendation 5: Develop and implement a plan to ensure that ELIS provides USCIS personnel with complete, accurate, and timely background vetting data to enable higher quality and more effective benefits adjudication decisions.

OIG Analysis of USCIS Comments We obtained written comments on a draft of this report from the Director of USCIS. We have included a copy of the comments in their entirety in appendix B. In the comments, the Director appreciated the OIG acknowledging the significant internal adjustments and corrective actions USCIS has made to the Transformation Program over the past year to ensure positive and significant www.oig.dhs.gov 41 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security progress towards enhancing the reliability, security, and efficiency of ELIS. Also, the Director indicated that USCIS remains committed to successfully implementing an electronic immigration services processing operation that is supported by a web-based, end-to-end adjudicative case management system. The USCIS Director concurred with all of our recommendations.

We reviewed the Director’s comments, as well as the technical comments previously submitted under separate cover, and made changes to the report as appropriate. Following is our evaluation of the USCIS Director’s response to each recommendation in the draft report provided for agency review and comment. Recommendation 1: Assess and address the ELIS training needs of USCIS field offices and service centers to ensure that ELIS users receive sufficient, hands-on training prior to each release. Management Comments The Director concurred with recommendation 1, stating that USCIS has already made considerable progress since April 2016 to ensure field office and service center personnel have sufficient training and knowledge to effectively use ELIS and its sub-systems to adjudicate N-400 applications.

Specifically, USCIS provided in-person and virtual training for pre-processing and adjudicative functionalities, and also posted training materials to an online dashboard for internal use. Further, USCIS provided guidance on ELIS case access and the use of peripheral computer equipment such as tablets and scanners. Most recently, USCIS began cross collaboration to identify training issues and gaps based on incident tickets submitted by users, and a Training Needs Analysis survey. This initiative allowed USCIS to gather targeted feedback from end-users, which helped in developing refresher training for future ELIS releases.

The outcome of this feedback led to refined N-400 training options, such as ongoing webinars and teleconferences, reference guides, and ad hoc instruction. USCIS is also developing an advanced ELIS N-400 training curriculum. The Director estimated that the new curriculum will be completed by December 31, 2017, and will be implemented on an ongoing basis as new system functionalities are deployed. OIG Analysis We agree that considerable progress has been made to assess and address the training needs of ELIS end-users.

We consider the actions described by the USCIS Director to be an effective approach to enhancing ELIS end-user training. We look forward to receiving updates on these actions as they are implemented. This recommendation will remain open and resolved until USCIS provides evidence that development of the new advanced training curriculum for ELIS end-users has been completed. www.oig.dhs.gov 42 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Recommendation 2: Perform a risk-based analysis of all unresolved ELIS technical issues to ensure that program resources are dedicated to addressing the highest risks impacting the integrity, operational efficiency, and delivery of customer service in immigration benefits processing. Management Comments The Director concurred with recommendation 2, stating that USCIS conducted a risk-based assessment of all ELIS N-400 product line technical issues and categorized corrective actions for the highest risk items into five areas.

We outlined four of the five risk areas in our Management Alert on USCIS N-400 processing, dated January 19, 2017. Additionally, the Director stated that USCIS introduced a portfolio-based framework that aligns Transformation efforts with specific business domains, while placing users at the center of product design, delivery, and testing. Portfolio teams work closely with the directorates to prioritize new requirements, emerging needs, and technical issues. This ensures that Transformation is consistently delivering high-value IT capabilities and business process improvements to satisfy objectives.

According to the Director, USCIS operational directorates conduct extensive user-based discovery and release planning as part of this approach. Specifically, field office and service center personnel partner with Transformation portfolio teams to inform requirements collection and refinement, co-design workflow processes and prototypes, and improve the prototypes through iterative testing and redesign. The field office and service center employees assist in defining and redefining problems, developing physical and digital solutions, and establishing development schedules. Features are prototyped and beta-tested with small groups of users in the field, and are continuously updated based on user feedback before deployment across the enterprise.

OIG Analysis We acknowledge the corrective actions taken to address the ELIS N-400 product line technical issues. Additionally, we recognize that USCIS restructured Transformation delivery efforts into portfolio teams to improve alignment with USCIS business operations, and assigned a portfolio manager to work with the designated business leads from each operational directorate. We consider the actions described by the USCIS Director as an effective approach to satisfying business objectives and mitigating Transformation risks. The actions taken satisfy the intent of this recommendation.

This recommendation is now closed. Recommendation 3: Develop and implement a plan for reducing ELIS technical debt as a means of improving current benefits processing automation and long-term architectural stability. www.oig.dhs.gov 43 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Management Comments The Director concurred with recommendation 3, stating that in the first quarter of FY 2017 USCIS developed and implemented a plan, allocating approximately 70 percent of team capacity toward addressing technical debt. The technical teams began a series of initiatives to increase ELIS’ availability and performance, and improve the quality and consistency of testing. Upon reducing the technical debt to desired levels, OIT will continue to devote approximately 20 percent of team capacity toward a combination of refactoring efforts including reduction of technical debt, resolution of functional defects, and implementation of system quality enhancements.

The Director remarked that technical debt reduction will be an ongoing activity throughout the program. Further, organizational realignment of the Transformation Program has involved updating the strategic direction and scope of the program to optimize the benefits of existing ELIS processing capabilities before undertaking new ones. The USCIS Director concluded that the realignment has made it easier to make significant improvements to the technical foundations of ELIS to minimize technical debt. OIG Analysis We agree that employing technical teams to work on a series of initiatives to reduce technical debt will improve benefits processing and long-term architectural stability.

We believe the actions the USCIS Director described constitute an effective approach to reducing ELIS technical debt. This recommendation will remain open and resolved until USCIS provides an estimated completed date. Recommendation 4: Clearly define agency-wide business goals and objectives for each benefit type automated in ELIS and monitor program performance against accomplishment of these objectives, rather than by release schedule. Management Comments The Director concurred with recommendation 4 stating that to clearly define agency-wide Transformation Program goals, USCIS’ Executive Coordination Council established a new set of eight business objectives for measuring the operational impacts of ELIS.

Specifically, these business objectives will help show how ELIS impacts adjudication time, reduces paper movement, and improves security. USCIS is currently analyzing data and obtaining stakeholder feedback to further define the appropriate qualitative and quantitative metrics for each of the business objectives. Additionally, the Director stated that the Transformation Program has incorporated a new management approach for ongoing and future ELIS development that includes improved program oversight, user feedback, and technical innovation. As such, USCIS can implement a technical www.oig.dhs.gov 44 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security development and procedural approach that ensures Transformation remains a results-focused digital service factory able to rapidly respond to evolving changes.

USCIS plans to complete these actions by December 31, 2017. OIG Analysis We agree that developing metrics for the eight business objectives will enable USCIS to more clearly measure the operational impacts of ELIS. Additionally, we consider the new management approach an effective means to ensure Transformation remains results-focused and able to respond to evolving changes. This recommendation will remain open and resolved until USCIS provides evidence that qualitative and quantitative metrics for each of the eight business objectives have been defined.

Recommendation 5: Develop and implement a plan to ensure that ELIS provides USCIS personnel with complete, accurate, and timely background vetting data to enable higher quality and more effective benefits adjudication decisions. Management Comments The Director concurred with recommendation 5, stating that USCIS has developed and implemented a plan to ensure that ELIS provides complete, accurate, and timely background vetting information. The Director stated that USCIS has made improvements to address deficiencies identified by the OIG related to three types of background security checks. Specifically, to ensure accuracy of the background checks, USCIS is performing quality assurance on 100 percent of the system-initiated background check inquiries related to naturalization cases.

All background checks performed in ELIS during naturalization pre-processing are also run through an external, legacy system interface. A new tool, Validation of ELIS Risk and Fraud Information, compares the results of background checks run through USCIS’ Active Tool for Linked Analysis and Screening and its Person-Centric Query System against results from Custom and Border Protection databases. Additionally, the Director stated that USCIS has resolved ELIS issues involving the handling of just-in-time background checks. Specifically, USCIS has incorporated a repetitive verification process that allows ELIS to receive information about new derogatory encounters while also making an additional last minute requests for new encounters just before oath ceremonies.

Finally, USCIS will continue to integrate background check services such as the new, mandated FBI Name Check Modernization web service to augment existing ELIS capabilities. This capability, which is targeted for implementation in October 2017, will conform to a modernized USCIS architecture and allow officers to readily see results. As the first subscriber to the FBI Name Check Modernization www.oig.dhs.gov 45 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Service, ELIS will display all submitted names and related biographic data to the FBI, along with the date of request and response. OIG Analysis We appreciate the improvements that USCIS has made to its background security check process.

In particular, we recognize that USCIS is now performing quality assurance on 100 percent of the system-initiated background security checks related to Naturalization cases. This recommendation will remain open and resolved until USCIS provides documented evidence that just-in-time background check deficiencies have been resolved, and that the FBI Name Check Modernization service has been implemented effectively so that ELIS displays all submitted names and related biographic data, including date of request and response. www.oig.dhs.gov 46 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Appendix A Objective, Scope, and Methodology As part of our ongoing responsibilities to assess the efficiency, effectiveness, and economy of departmental programs and operations, we audited USCIS' transition to electronic processing for the Form N-400, Application for Naturalization. Specifically, our objective was to determine the effectiveness of USCIS’ efforts to automate the Form N-400, Application for Naturalization. This was a follow-up to two previous reports, USCIS Automation of Immigration Benefits Processing Remains Ineffective (OIG-16-48) and Better Safeguards Are Needed in USCIS Green Card Issuance (OIG-17-11).

We researched and reviewed Federal laws and agency guidance, policies, and procedures related to Naturalization processing, immigration benefits delivery, and IT systems implementation. We obtained published reports, documents, testimony, and news articles regarding USCIS’ automated processing and benefits delivery. Additionally, we reviewed published GAO and DHS OIG reports to identify prior findings and recommendations relevant to this audit. We used this information to establish a data collection approach that consisted of interviews with USCIS stakeholders, focused information gathering, documentation analysis, site visits, and system demonstrations to accomplish our audit objectives.

We observed N-400 processing in ELIS, application submission in myUSCIS, and security patch management practices for ELIS. We held more than 60 meetings and teleconferences with more than 200 USCIS staff at headquarters and at field offices. At headquarters, we met with officials and personnel from the following offices: Office of Transformation Coordination, Office of Management Directorate, Office of Information Technology, Field Operations Directorate, Fraud Detection and National Security, and Customer Service and Public Engagement Directorate. We also visited the National Benefits Center in Missouri, as well as USCIS Field Offices in California, Maryland, New Jersey, and New York in January and February 2017.

During our field visits, we met with executive personnel, Section Chiefs, Immigration Services Officers, and ELIS end-users to understand operational use in the field. We discussed USCIS’ N-400 processing, end-user training, end-user testing, general ELIS system use, technical support, operational metrics, reporting, and communication with headquarters. We collected supporting documentation from USCIS and received system demonstrations of ELIS and its peripheral systems, and observed several naturalization interviews. We conducted technical testing at USCIS headquarters to review the deployment of security patches on operating systems and the security configuration of the ELIS 2 website and supporting back-end database.

This www.oig.dhs.gov 47 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security security testing provided USCIS management and OIG with an assessment of the effectiveness of information security controls on USCIS computer resources and information. We conducted this performance audit between December 2016 and April 2017, pursuant to the Inspector General Act of 1978, as amended, and according to generally accepted government auditing standards. Those standards require that we plan and perform the audit to obtain sufficient, appropriate evidence to provide a reasonable basis for our findings and conclusions based upon our audit objectives. We believe the evidence obtained provides a reasonable basis for our findings and conclusions based upon our audit objectives. www.oig.dhs.gov 48 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Appendix B USCIS Comments to the Draft Report www.oig.dhs.gov 49 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security www.oig.dhs.gov 50 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security www.oig.dhs.gov 51 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security www.oig.dhs.gov 52 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security www.oig.dhs.gov 53 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security www.oig.dhs.gov 54 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Appendix C Status of OIG and GAO Prior Recommendations Report Recommendation Current status USCIS Faces Challenges in Modernizing Information Technology, OIG-05-41, September 2005, reported that the USCIS IT environment for processing immigration benefits was inefficient, hindering its ability to carry out its mission.

1. Develop a modernization strategy that includes short- and long-term goals, funding plans, and Closed performance measures to guide USCIS entities in accomplishing their citizenship and immigration services missions. 2. Complete implementation of plans to centralize IT by placing all USCIS IT employees, budgets, Closed and systems under the CIO’s authority and control.

3. Ensure that the centralized CIO operation and its IT transformation plans and systems initiatives Closed are linked to and effectively support the consolidated USCIS strategy. 4. Review, analyze, and reengineer benefits adjudication activities to help eliminate duplication, Closed transition from paper-based processes, better integrate systems, and provide systems access to the users who need it.

5. Finalize and implement plans to upgrade and standardize IT hardware and software systems to support reengineered processes and systems integration and access improvement initiatives. Closed 6. Ensure representation and participation of users from across USCIS in all process reengineering Closed and IT transformation activities.

U.S. Citizenship and Immigration Services’ Progress in Modernizing Information Technology, OIG-07-11, November 2006, reported that that the agency had not finalized its transformation or acquisition approach, completed technology upgrades, or increased stakeholder involvement. 1. Develop a modernization strategy that includes short- and long-term goals, funding plans, and Closed performance measures to guide USCIS entities in accomplishing their citizenship and immigration services missions. 2.

Complete implementation of plans to centralize IT by placing all USCIS IT employees, budgets, Closed and systems under the CIO’s authority and control. 3. Ensure that the centralized CIO operation and its IT transformation plans and systems initiatives Closed are linked to and effectively support the consolidated USCIS strategy. 4.

Review, analyze, and reengineer benefits adjudication activities to help eliminate duplication, Closed transition from paper-based processes, better integrate systems, and provide systems access to the users who need it. 5. Finalize and implement plans to upgrade and standardize IT hardware and software systems to support reengineered processes and systems integration and access improvement initiatives. Closed 6.

Ensure representation and participation of users from across USCIS in all process reengineering Closed and IT transformation activities. U.S. Citizenship and Immigration Services’ Progress in Modernizing Information Technology, OIG-09-90, July 2009, reported that USCIS had established Transformation Program governance, but made limited progress due to ineffective planning, incomplete process reengineering, and inconsistent stakeholder participation. 1. Develop an updated transformation approach, strategy, or plan to communicate end-state business Closed processes and IT solutions to stakeholders.

2. Develop and implement a plan to achieve sufficient and consistent stakeholder participation in Closed process reengineering and requirements definition activities. 3. Complete evaluations to document the results and lessons learned from the pilot and proof-of- Closed concept programs. www.oig.dhs.gov 55 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Report Recommendation 4.

Develop a USCIS Office of Information Technology staffing plan that includes specific actions and milestones for recruiting and retaining fulltime employees. Current status Closed 5. Communicate guidelines and procedures for acquiring, developing, and managing IT solutions, as Closed defined by the DHS and USCIS CIOs, to stakeholders. 6.

Provide the CIO agency-wide budget and investment review authority for all USCIS IT initiatives Closed and system development efforts. U.S. Citizenship and Immigration Services’ Progress in Transformation, OIG-12-12, November 2011, reported that Transformation implementation was delayed and USCIS was relying on paper-based processes to support its mission. 1. Complete business and technology process documentation to provide the detail necessary to implement the transformation program effectively.

2. Revise its current governance structure to enable more streamlined program decision making. 3. Ensure that transformation program staff possesses the necessary skills to implement the Closed Closed Closed transformation program.

U.S. Citizenship and Immigration Services Information Technology Management Progress and Challenges, OIG-14-112, July 2014,reported that the agency still did not have technology systems in place to support mission needs, causing delays to benefits processing. 1. Finalize and communicate USCIS’ IT Strategic Plan to ensure that IT supports the mission of Resolved/Open USCIS and the Department. 2.

Develop and implement a plan of action and milestones to address senior level staffing vacancies including Chief of Staff, Chief Technology Officer, and Chief, Strategic Vendor Management. Closed 3. Coordinate with the owners of ELIS and the Electronic Document Management System to ensure Closed users are provided with adequate training. 4.

Develop and communicate a plan of action and milestones to refresh outdated IT infrastructure, Closed including computers, printers, and software.

USCIS

Automation of Immigration Benefits Processing Remains Ineffective, OIG-16-48, March 2016, reported that only 2 of 90 immigration benefit types were available for online customer filing. Further, the ELIS approach did not ensure stakeholder involvement, performance metrics, system testing, or user support needed for ELIS to be effective. 1. Ensure adequate communications and stakeholder involvement throughout system development Resolved/Open and deployment so that each ELIS release provides needed functionality.

2. Develop and implement performance metrics to measure operational efficiencies achieved via Resolved/Open automation of each benefit type in ELIS. 3. Develop and implement a plan for end-user involvement in end-to-end testing to ensure each ELIS Resolved/Open release functions as required prior to deployment.

4. Develop and implement a plan to provide adequate support for addressing system issues and Resolved/Open assisting end-users following deployment of each ELIS release. Better Safeguards Are Needed in USCIS Green Card Issuance, OIG-17-11, November 2016, reported that system errors in ELIS had hindered proper Green Card issuance. At least 19,000 cards were issued with incorrect information or in duplicate over the preceding 3 years and USCIS efforts to remediate and recover these cards were inadequate.

1. Ensure ELIS design and functionality problems are corrected to prevent, to the extent possible, Resolved/Open further Green Card processing errors. 2. Ensure development and implementation of the internal controls needed to ensure Green Card errors are identified and corrected early in the production process, prior to card issuances.

Resolved/Open 3. Ensure development and implementation of a standard process for card recovery efforts. Resolved/Open 4. Ensure development and implementation of a standard procedure for identifying and preventing Resolved/Open www.oig.dhs.gov 56 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Report Recommendation unrecoverable cards from being used.

Current status 5. Implement a centralized method to track and document Green Cards that are returned through recovery efforts. Resolved/Open 6. Complete and implement identity-proofing capability to enable customers to submit address Resolved/Open changes online in ELIS.

7. Evaluate the costs and benefits of using USPS’ Signature Confirmation as an alternative secure Resolved/Open method for delivering Green Cards to applicants. Management Alert—U.S. Citizenship and Immigration Services’ Use of the Electronic Immigration System for Naturalization Benefits Processing, OIG-17-26-MA, January 2017, reported a range of system deficiencies that have slowed the processing and productivity of naturalization processing. 1.

Ensure the four minimal requirements of the Field Operations Directorate are met prior to Resolved/Open. returning to ELIS processing of N-400 naturalization applications. 2. Performa a risk-based analysis of all unresolved ELIS technical issues to ensure that, going Resolved/Open forward, all system improvement decisions are based on potential agency operational impact and risk to public safety.

USCIS

Transformation: Improvements to Performance, Human Capital, and Information Technology Management Needed as Modernization Proceeds, GAO-07-1013R, July 2007, reported that USCIS’ transformation plans lacked adequate stakeholder involvement as well as performance metrics for the organization. 1. Document specific performance measures and targets for the pilots, increments, and the Closed transformed organization that are outcome-oriented, objective, reliable, balanced, limited to the vital-few, measurable, and aligned with organizational goals. 2.

Increase coordination between program office and the Office of Human Capital to ensure transformation and human capital change initiatives are aligned. 3. Plan for the number and types of human resources required in the program office to carry the transformation through 2012. 4.

Plan for obtaining and developing the IT human capital necessary to support the transformation. 5. Determine the critical skills and competencies that will be needed to achieve future programmatic results as well as strategies to address gaps in employee numbers, deployment, and skills and competencies. Closed Closed Closed Closed 6.

Address continuity in key transformation leadership positions and address impacts to time frames Closed when key personnel leave. 7. Use performance expectations and competencies to hold USCIS executives and employees accountable for achieving the goals of the transformation. 8.

Continue to develop an enterprise architecture that sufficiently guides and constrains the transformation plans, as DHS works to address limitations in its own enterprise architecture and alignment processes. 9. Complete a comprehensive communication strategy that involves communicating early and often to build trust, ensuring consistency of message, and encouraging two-way communication. Further, the communication strategy should address plans for communicating implementation goals and timelines to demonstrate progress.

Closed Closed Closed 10. Complete a comprehensive communication strategy that addresses plans for formally engaging Closed internal and external stakeholders throughout the transformation, and tailors information to meet these stakeholders’ specific needs. 11. Complete a comprehensive communication strategy that addresses plans for a long-term, detailed Closed strategy to share information with employees and stakeholders over the course of the transformation. www.oig.dhs.gov 57 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Report Recommendation 12.

Document specific performance measures and targets for the pilots, increments, and the transformed organization that are outcome-oriented, objective, reliable, balanced, limited to the vital-few, measurable, and aligned with organizational goals. Current status Closed Immigration Benefits: Consistent Adherence to DHS's Acquisition Policy Could Help Improve Transformation Program Outcomes, GAO-12-66, November 2011, identified gaps in USCIS’ plans that created risks that could undermine its successful implementation of the Transformation Program. 1. Develop and maintain an Integrated Master Schedule consistent with these same best practices for Closed the Transformation Program.

2. Ensure that the life-cycle cost estimate is informed by milestones and associated tasks from Closed reliable schedules that are developed in accordance with the nine best practices we identified. Immigration Benefits System: Better Informed Decision Making Needed on Transformation Program, GAO-15-415, May 2015, reported that the changes to the program acquisition strategy significantly delayed the program’s planned schedule. 1.

Re-baseline cost, schedule, and performance expectations for the remainder of the Program. 2. Ensure that the Acquisition Review Board is effectively monitoring the Transformation Program’s performance and progress toward a predefined cost and schedule; ensuring that corrective actions are tracked until the desired outcomes are achieved; and relying on complete and accurate program data to review the performance of the Transformation Program against stated expectations. 3.

Ensure that the Executive Steering Committee is effectively monitoring the Transformation Program’s performance and progress toward a predefined cost and schedule and relying on complete and accurate program data to review the performance of the Transformation Program against stated expectations. 4. Direct the department’s Chief Information Officer to use accurate and reliable information, such as operational assessments of the new architecture and cost and schedule parameters approved by the Under Secretary of Management. Closed Open Open Open Immigration Benefits System: U.S. Citizenship and Immigration Services Can Improve Program Management, GAO-16- 467, July 2016, reported that the Transformation Program has experienced management challenges.

1. Complete planning for software releases prior to initiating development and ensure software meets Open business expectations prior to deployment. 2. Consistently implement the principles of the framework adopted for Agile software development.

3. Define and consistently execute appropriate roles and responsibilities for individuals responsible for development activities consistent with its selected development framework. 4. Identify all system users and involve them in release planning activities.

5. Write user stories that identify user roles, include estimates of complexity, take no longer than one sprint to complete, and describe business value. 6. Establish outcomes for Agile software development.

7. Monitor program performance and report to appropriate entities through the collection of metrics. 8. Conduct unit and integration, and functional acceptance tests, and code inspection consistent with stated program goals.

9. Develop complete test plans and cases for interoperability and end-user testing, as defined in the USCIS Transformation Program Test and Evaluation Master Plan, and document the results. 10. Clearly define measures against to analyze differences between services expected delivered.

11. Ensure contracting officer representatives are maintaining complete contract files. 12. Ensure quality assurance surveillance plans are developed when appropriate.

Open Open Open Open Open Open Open Open Closed Open Closed www.oig.dhs.gov 58 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Appendix D: Major ELIS Releases 2012 to 2016 SYSTEM Benefit Types or Services Release A2.1 Form I-539, Application to Extend/Change Nonimmigrant Status ELIS 1 Release A2.3 USCIS Immigrant Fee Payment Release A2.4 Form I-526, Immigrant Petition by Alien Entrepreneur Release 5.01 Form I-90, Application to Replace Permanent Resident Card Release 6.1 Form I-765, Application for Employment Authorization Form I-821D, Consideration of Deferred Action for Childhood Arrivals Release 7.1 Form N-400, Application for Naturalization Release 6.2 Form I-821, Application for Temporary Protected Status Release 8.1 Form N-336, Request for a Hearing on a Decision in Naturalization Proceedings Under Section 336 Form N-565, Application for Replacement Naturalization/Citizenship Document Release 9.1 Payment for Form I-131A, Application for a Travel Document ELIS 2 (ELIS) Release Date 5/22/2012 Status 42 Decommissioned June 2015 5/18/2013 7/21/2013 Decommissioned/ replaced by ELIS 2 August 2015 Decommissioned June 2015 3/30/2015 Operational 12/5/2015 Operational 4/13/2016 Operational for select applications43 5/16/2016 Operational 8/26/2016 Operational 9/30/2016 Operational  42A total of 11 products or services were released in ELIS; nine were operational as of April 2017. 43At the time of this review, ELIS was used to process applications received between April and August 2016, as well as a limited number of e-file applications. www.oig.dhs.gov 59 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Appendix E: USCIS ELIS Interfaces as of April 2016   InternalSystemInterfaces 1.IdentityCredentialAccessManagement(ICAM) 2.NationalAppointmentSchedulingService(NASS) 3.EnterpriseCorrespondenceHandlingOnline (ECHO) 4.LocalPrinting 5.NoticePrinting 6.EnterprisePrintManagerService/Notice GenerationSystem(EPMSNGS) 7.CustomerProfileManagementSystem(CPMS) 8.CustomerRelationshipInterfaceSystem(CRIS) 9.CPMSSupportService 10.PersonCentricQuerySystem(PCQS) 11.EnterpriseServiceBusVerificationService(ESB VS) 12.TransformationSupportService(TSS) 13.ApplicationSupportCenter(ASC) 14.BenefitsBiometricsSupportSystem(BBSS) 15.NationalFileTrackingSystem(NFTS) 16.CentralIndexSystem(CIS)     17.EnterpriseCitizenshipandImmigrationServices CentralizedOperationalRepository(eCISCOR) 18.StandardManagementAnalysisReportingTool (SMART)/Reporting  ExternalSystemInterfaces 1.FBINameCheck 2.Pay.govTrustedCollectionService(TCS) 3.CollectionInformationRepository(CIR) 4.USPS 5.Pay.gov 6.JPMorgan/Chase(JPMC) 7.CustomsandBorderProtection(CBP)/TECS 8.FBIIntegratedAutomatedFingerprint IdentificationSystem(IAFIS) 9.AutomatedBiometricIdentificationSystem (IDENT) 10.ENFORCEIntegratedDatabase(EID) 11.ExecutiveOfficeofImmigrationReform(EOIR) www.oig.dhs.gov 60 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Appendix F ELIS Performance Measures and Results as of July 2016 Key Performance Parameters Threshold Objective  1 2 3 4 5 6 Account Accuracy ELIS shall establish only one account per identical set of key biographic and biometric data when applicable. Interoperability ELIS shall successfully support data transmission to/from the internal USCIS systems and external agency systems in accordance with interface agreements. ELIS Reliability ELIS shall provide service to end- users and successfully respond to interfaces without interruption.

ELIS System Availability ELIS shall allow for high System Availability covering operations 24/7 for external and internal customers. ELIS Maintainability ELIS shall promptly restore services due to unexpected outage. ELIS Scalability ELIS shall have the ability to support future growth to meet rising demand. 99.97% 100% 90.04% 99.97% Actual 99.74% Below Threshold Immigrant: 63.83% Humanitarian:73.68% Citizenship: 84.85% Below Threshold 641 hours 712 hours 105.37 hours Below Threshold 97.63 % 98.88% 99.14% Exceeds Objective No more than 10 hours No more than 8 hours 0.91 hours Exceeds Objective 95% transactions per year 99.97% transactions per year 100% Meets Objective 7 Manage Case Disposition 95% 100% 100% ELIS shall support processing and adjudication of USCIS Lines of Business.

8 Support Workload and Operational Performance ELIS shall gather and submit information to USCIS enterprise data warehouse that supports decisions on workload allocations and performance. Meets Objective 95% 100% 100% Meets Objective www.oig.dhs.gov 61 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Appendix G ELIS Vulnerability Assessment We identified four operating system and three database vulnerabilities on the ELIS system. Although these issues were present during our assessment, we expect they will be resolved during the standard patch management process currently implemented by the ELIS team. Overall, we determined that USCIS has implemented a patch management program which deploys software patches to reduce vulnerabilities on servers and databases present within ELIS.

Methodology and Analysis We conducted security control configuration and patch management scans using specialized software on the supporting servers and databases within USCIS’ ELIS system. For example, our software will identify missing patches on operating systems and software from third-party companies. Additionally, we conducted scans of the ELIS system to identify any unsupported operating systems. Unsupported operating systems present a significant risk to government systems because the developer is no longer releasing updates to address any critical security flaws.

The following analysis is based on the results of our assessments. We scanned 57 servers and 1 database within the USCIS ELIS accreditation boundary. Overall, we identified 2 unique critical vulnerabilities, and 2 unique high risk vulnerabilities. The critical vulnerabilities we identified are processes with broken links to their original executable files and out-of-date virus scan definitions.

High vulnerabilities identified are an out-of-date anti-virus software platform and out-of-date installations of Oracle’s Java. After analyzing our database assessments, we determined there were 3 high risk vulnerabilities related to configuration controls. However, making changes to resolve these vulnerabilities could prevent the database from working as intended. As a result, these vulnerabilities should be addressed only if it would not adversely affect the operation of the ELIS system.

Finally, we performed scans to identify any unsupported operating systems still currently in use throughout the ELIS system. After reviewing our scans, we did not identify any unsupported operating systems. www.oig.dhs.gov 62 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Appendix H Office of IT Audits Major Contributors to This Report Kristen Bernard, Director Kristen Fogarty, Audit Manager Gregory Flatow, Program Analyst Theresa Lowell, Program Analyst Thomas Rohrback, Chief, Information Assurance and Testing Branch David Bunning, Information Technology Specialist Steven Staats, Referencer Tarsha Cary, Referencer www.oig.dhs.gov 63 OIG-18-23 OFFICE OF INSPECTOR GENERAL Department of Homeland Security Appendix I Report Distribution Department of Homeland Security Secretary Deputy Secretary Chief of Staff Deputy Chiefs of Staff General Counsel Executive Secretary Director, GAO/OIG Liaison Office Assistant Secretary for Office of Policy Assistant Secretary for Office of Public Affairs Assistant Secretary for Office of Legislative Affairs Director, USCIS Deputy Director, USCIS Chief Information Officer, USCIS Liaison, USCIS Office of Management and Budget Chief, Homeland Security Branch DHS OIG Budget Examiner Congress Congressional Oversight and Appropriations Committees, as appropriate www.oig.dhs.gov 64 OIG-18-23 Additional Information and Copies To view this and any of our other reports, please visit our website at: www.oig.dhs.gov. For further information or questions, please contact Office of Inspector General Public Affairs at: [email protected]. Follow us on Twitter at: @dhsoig.

OIG Hotline  To report fraud, waste, or abuse, visit our website at www.oig.dhs.gov and click on the red "Hotline" tab. If you cannot access our website, call our hotline at (800) 323-8603, fax our hotline at (202) 254-4297, or write to us at: Department of Homeland Security Office of Inspector General, Mail Stop 0305 Attention: Hotline 245 Murray Drive, SW Washington, DC 20528-0305