Technology Control Plan¶
Center for Computational Innovations, Rensselaer Polytechnic Institute
This document describes the control environment in place at the Center for Computational Innovations, a high-performance computing facility owned and operated by Rensselaer Polytechnic Institute. The Center for Computational Innovations, or CCI, is located in the Rensselaer Technology Park at 405 Jordan Road in the town of North Greenbush, New York.
2 IT General Control Areas¶
2.1 Computer Operations¶
2.1.1 Backup and Recovery Procedures¶
The CCI is a research facility. It is not intended for production workloads. Individuals using the facility are responsible for the protection of their own data. The CCI has no formal processes nor procedures for backup and recovery for the benefit of end-users.
2.1.2 Environmental Controls¶
Humidity and cooling regulation for the facility is managed by an automated building control system. Status information is relayed by the building control system to Physical Facilities staff on the main campus and to the Public Safety personnel.
Power systems, including the uninterruptable power system, is separately monitored by automated controls. These controls relay status information to Physical Facilities staff on the main campus.
The emergency generator for the CCI is tested bi-weekly.
2.1.3 Risk Assessment¶
The CCI information systems are periodically scanned for common and new vulnerabilities. Any vulnerability not documented will be risk assessed and documented.
2.2 Change Management¶
2.2.1 Patch Management¶
Subject to CCI review for relevance, CCI will update information system protection mechanisms within 7 days of new releases for critical remote security issues, 30 days for high severity, 60 days medium severity, and 180 days for all other applicable updates. Emergency updates are applied as soon as practicable.
2.3 Code & Document Version Control¶
Documentation is available through wikis. A public facing version is available for systems users and an internal one is used by the systems staff. They are managed using a GitHub repository, mkdocs, and MediaWiki. Version control is implemented. Trouble tickets are managed through a Zendesk instance.
2.4 Asset Management¶
2.4.1 Data and Asset Classification¶
Data and asset classification follow Institute and DotCIO policies. In the case of the AI Hardware Testbed, data and asset classification adheres to the IBM Research AI Hardware Center Master Agreement, including not allowing any PI (personally identifiable) information.
2.4.2 Hardware Inventory¶
The AI Testbed Hardware inventory will be kept and updated annually. Information stored is serial number, model number, model type.
2.4.3 Software Inventory¶
The software inventory of all packages and vendor software that are installed and maintained center-wide will be reviewed and updated annually. Stored inventory data will include software package data: package, version number. Obsolete services will be scheduled for sun-setting annually.
2.5 Logical Access¶
2.5.1 User Access Provisioning¶
User access is organized around a project concept. Projects correspond to research endeavors or other focused activity. Projects are requested by project sponsors. User accounts are requested by the individuals, but an account is not created and associated with a project until the project sponsor designates the individual as a project participant. As part of the user account request, the requesting individual must accept the Rensselaer User Responsibility agreement for the CCI and the CCI Acceptable Use Policy.
The procedures and forms for requesting projects and user accounts and the referenced policies are documented on the CCI wiki. Project request forms and user account request forms are kept on file.
2.5.2 User Access De-provisioning¶
User access to a project is terminated when either the project sponsor requests the access be terminated or the project itself is terminated. User access may be suspended if the account has been inactive for more than 90 days.
2.5.3 Periodic Access Reviews¶
Project sponsors are responsible for reviewing who has access to the project. Project sponsors are provided with a list of project participants at regular intervals and on request.
2.5.4 Password Requirements¶
Where passwords are used as access credentials, the following complexity requirements are enforced by the password change mechanism:
- Must contain a lowercase letter ([a-z]).
- Must contain an uppercase letter ([A-Z]).
- Must contain a digit ([0-9]).
- Must contain punctuation (anything not alphanumeric).
- Must be at least 8 characters in length.
Two-factor authentication is implemented through the use of the Google Authenticator and is required for all user access.
2.5.5 Access to Computational Resources¶
The CCI recognizes four major classes for projects based on their sponsorship. These classes are (1) projects that are Rensselaer sponsored for Rensselaer personnel and affiliates, (2) projects that are SUNY and SUNY Poly sponsored for SUNY and SUNY Poly interests, (3) that are IBM sponsored for IBM employees and corporate affiliates, and (4) IBM Research Artificial Intelligence Hardware Center sponsored for the AI Hardware Center membership. Other classes may be included based on approval by the Governance Board.
The automated job scheduler (SLURM) attempts to the best of its abilities to assign computational resources to the available work load to achieve resource allocation on a high-priority basis among the four classes as 20%-10%-20%-50%, respectively, as a rolling average basis. (If additional classes are defined as provided for above, then the number of classes and distribution of resources are adjusted accordingly.) Unused resource allocations by one class are forfeited and made available to others. Changes to allocation made with Governance Board approval.
The behavior of the job scheduler in managing resource allocation is reviewed monthly for overall fairness and efficiency.
Management of allocations within a project class is with the approval of the individual project sponsor.
2.5.6 Access to File Resources¶
The CCI storage architecture is based on the General Purpose File System, GPFS. GPFS implements an access control list mechanism to limit access to applicant and to data assets. When projects are created access control lists are defined for project-specific storage areas that limit access to project members. Thereafter, project sponsors are responsible for access control list management and ensuring appropriate access restrictions for project data and applications.
2.5.7 Session Controls¶
The CCI is a partially-closed computing environment. All user access to CCI systems is through designated landing pads or through a proxy server from previously whitelisted sites. The other CCI resources (including the high-performance computing clusters and the file storage system) are accessible only via the landing pads. Outbound connections are not allowed except of previously authorized whitelisted sites. (CCI management approves all whitelisted sites.) All connections to or from the CCI network require encryption.
Network access is automatically blocked after a sequence of five failed attempts to login. Failed login attempts are logged. The CCI systems do not inform the user of any specific reason for a login failure (e.g., incorrect password versus username). User accounts may be locked if they have not been used in the past 90 days.
Upon logging into the CCI systems, users are presented with a "splash screen" reminding them of the CCI Acceptable Use Policy and other important conditions related to their usage of CCI resources. The following text is displayed:
CCI SSH Gateway (Landing pad) Please report all support and operation issues to firstname.lastname@example.org On-line documentation for the systems can be found at: https://secure.cci.rpi.edu/wiki CCI does not provide any data backup services. Users are responsible for their own data management and backup. Use is subject to the terms of the policy for Acceptable Use of CCI Resources.
2.5.8 Privileged User Accounts¶
CCI audits all privileged functions, and controls access using access control lists based on identity or role. Sudo and UAC are deployed were applicable to limit access to root type accounts.
2.5.9 Audit logs¶
CCI creates, protects, and retains the following information system audit records:
- System logs – aggregated to a common syslog server and kept for one year
- SLURM logs – kept indefinitely
- Email logs – kept for 90 days
- Webserver logs – kept for 90 days
All systems synchronize their clocks from a common NTP synchronized time source.
2.6 Incident Management¶
CCI personnel are bound to following the Rensselaer incident response plan. Any suspected information technology security event must be reported to the IT Support Services help desk for handling. From that point forward, the Rensselaer incident response plan dictates how the event is investigated and whether and to what extent CCI staff will be included in the investigation.
The details of the Rensselaer incident response plan are beyond the scope of this document.
2.6.1 Security Incident Handling¶
If Rensselaer discovers or is notified of a breach or potential breach of security relating to CCI end-user information, Rensselaer will promptly notify the CCI end-user of such breach or potential breach within 72 hours of the discovery of the potential breach. If the breach or potential breach resulted from a failure or weakness in systems or procedures that were Rensselaer responsibility, Rensselaer shall make reasonable adjustments to systems and procedures to provide satisfactory assurances the breach or potential breach will not recur.
2.7 Problem Management and Technical Support¶
All problem reports and requests for assistance are routed through the Institutes IT Services and Support Center, ITSSC. The ITSSC relies on ZenDesk for ticketing, assignment, management, and monitoring of all requests and trouble reports.
2.8 Disaster Recovery¶
The Center for Computational Innovations will continue to operate on a "best effort" basis during any event resulting in the shutdown of the Rensselaer campus. Remote access to all CCI systems will remain operational for all users and system staff. Systems monitoring and biweekly maintenance will continue on a remote-only basis. All other non-critical maintenance items will be deferred until they are allowed to be completed on-site per Institute health safety protocols. Critical system maintenance will be conducted as soon as possible and in a manner that meets Institute health safety protocols, including external vendor support. During this time, we may also need to throttle large-scale data transfers to lessen the impact on core-campus networking and keep available sufficient bandwidth for campus lecture and remote classroom instructional activities.
Target estimate for maximum time the facility may be unavailable is two to five days.
2.9 Network Security¶
The CCI environment is logically isolated from any publicly accessible systems or networks. The CCI environment only allows access via landing pads and whitelisted outbound connections via proxy server. Rensselaer's DNS service incorporates security filtering via the Cisco Umbrella service.
CCI will receive security alerts, advisories, and directives from external sources include US-CERT, and disseminate this information to individuals with need-to-know in the organization. In the event of alerts, advisories, or directives that have widespread impact on the organization, internal security directives will be disseminated directly to information system users, managers, and administrators.
The CCI information system end points implement cryptography protocols to prevent the disclosure of data or credentials during transmission. IBM Security Key Lifecycle Manager (SKLM) is available and can provide encryption of data at rest. Certificates used for service validation and are presented to end users must be issued by a valid certificate authority that is included by default in common browsers, including systems used internally by admins or for inter service communication, but self-signed certs are acceptable.
The following encryption protocols are acceptable:
- AES with minimum key length of 128 bits
- ChaCha20 (RFC 7539)
- CAST (RFC 2144) with minimum key length of 128 bits
- ED25519 (RFC 7748)
- RSA (Rivest-Shamir-Alderman) with minimum key length of 2048 bits
- ECDSA (Elliptic Curve Digital Signature Algorithm) with minimal key length of 224 bits
- SHA-2 (Secure Hash Algorithm) with or without salt
- HMAC (RFC 2104)
- Poly1305 (RFC 7539)
2.10 Physical Security¶
The Center for Computational Innovations, the CCI, is at 405 Jordan Road in the Rensselaer Technology Park in the town of North Greenbush, New York. The sensitive technology—the computing systems and storage—are located in room 114 of the CCI.
2.10.2 Physical Security of the Facility¶
All exterior doors to the CCI facility are secure requiring an authorized RFID card for entry. The building is monitored by closed-circuit surveillance cameras located at strategic points on the building periphery and within the building itself. Camera video is routed to a video management system operated by Rensselaer's Office of Public Safety. Surveillance video is retained for no less than thirty days.
In addition to the surveillance system monitoring the facility, it is lightly staffed during normal business hours. In addition, computer operators from the main campus visit the facility for "walk-through" inspections at beginning and end of the third shift.
2.10.3 Physical access to the Facility¶
Physical access is via RFID card and is limited to CCI staff and Rensselaer maintenance personnel on a need-for-access basis.
Rensselaer's Office of Public Safety administers the access control system. The CCI operations manager authorizes Public Safety to activate and deactivate card access to the facility. The CCI operations manager requests from Public Safety a listing of all personnel with access to the CCI facility. The listing is reviewed for correctness.
Physical assess by all other individuals is screened. Entry into the facility itself requires a door be temporarily unlocked by CCI staff.
2.10.4 Physical Security of Room 114¶
Both entry doors to room 114 are secure requiring an authorized RFID card for entry. The doors are monitored by closed-circuit television cameras within the room. Camera video is routed to a video management system operated by Rensselaer's Office of Public Safety. Surveillance video is retained for no less than thirty days.
2.10.5 Physical access to Room 114¶
Physical access is via RFID card to room 114 and is limited to computer operators, systems administrators, and Rensselaer maintenance personnel on a need-for-access basis who are citizens of the United States of America or who have permanent resident alien status within the United States.
Rensselaer's Office of Public Safety administers the access control system. The CCI operations manager authorizes Public Safety to activate and deactivate card access to room 114. The CCI operations manager requests from Public Safety a listing of all personnel with access to room 114. The listing is reviewed for correctness.
Physical assess by all other individuals is by invitation and requires an escort by an authorized individual for the entire time the other individuals occupy room 114.
Fire Detection and Suppression Systems¶
All walls, doors, floor and ceiling meet required codes for fire rating. The large glass viewing windows are supplemented with metal screens to meet code. The screens drop into place when any fire or smoke sensor trips or pull-station is activated or when there is a loss of electrical power.
Room 114 has three active fire detection systems. An air sampling system continually monitors air from above and below the equipment in the room. Upon detecting significant particulate matter in the air samples (e.g., smoke), an emergency power-off mechanism is trigged to drop all electrical power to the room.
A set of detectors above and below the room 114 raised floor monitor the room for smoke and fire. These detectors are set to trigger an emergency power-off mechanism and to activate a clean agent (FM-200) fire suppression system.
A second, independent set of detectors above and below the room 114 raised floor also monitor the room for smoke and fire. These detectors are set to trigger an emergency power-off mechanism and to charge the water deluge system. (The individual water sprinklers in the room still would require activation by the heat from a fire before water would be released into the room.)
All three systems tie into the facility fire panel. The panel reports unusual status to Rensselaer's Office of Public Safety on the Troy campus; in the event of a fire alarm, the panel automatically signals the local fire department for emergency services.
CCI Technology Control Plan 1 April, 2021