--- layout: handbook-page-toc title: "IT Ops" --- ## On this page {:.no_toc .hidden-md .hidden-lg} - TOC {:toc .hidden-md .hidden-lg} ## Welcome to the IT Ops Handbook The IT Ops department is part of the [GitLab Business Ops](/handbook/business-ops/) function that guides systems, workflows, and processes and is a singular reference point for operational management. ## Get in Touch * [IT Ops Issues](https://gitlab.com/gitlab-com/business-ops/itops/issue-tracker/issues) (internal) * #it_help on Slack (internal) - This channel is used to communicate your hardware, software, and access issues to the IT Helpdesk team for triaging. * #sysadmin-it on Slack (internal) - This channel is used to communicate issues related to laptops and access requests. ## Mission Statement IT Ops will work with Security, the People Group, and Business Operations to develop automated on-boarding and off-boarding processes. We will develop secure integrations between Enterprise Business Systems and with our Data Warehouse. We will develop tooling and process to facilitate end-user asset management, provisioning and tracking. We will work to build API Integrations from the HRIS to third party systems and GitLab.com. We triage IT related questions as they arise. We build and maintain cross-functional relationships with internal teams to champion initiatives. We will spearhead on-boarding and off-boarding automation efforts with a variety of custom API integrations, including GitLab.com and third-party resources, not limited to our tech-stack, with scalability in mind. ## Access Requests * For information about the access request policies and security guidelines, please refer to the Security Team's [access request handbook page section](/handbook/engineering/security/#access-management-process). * For links to role based access request templates, system access templates, and other general instructions and FAQs, please refer to the [Access Requests page](/handbook/business-ops/it-ops-team/access-requests/). * For a diagram of how baseline entitlements work and what systems you must request to access other systems, [please refer to the diagram page.](/handbook/business-ops/it-ops-team/access-requests/baseline_flow/) ### Baseline & Role-Based Entitlements For information about baseline entitlements and role-based access, please refer to the [baseline entitlements handbook page](/handbook/engineering/security/#baseline-role-based-entitlements-access-runbooks--issue-templates). For information on how to create a Role-Based Entitlement, please refer to the [instructions on how to create role-based entitlements](/handbook/business-ops/it-ops-team/access-requests/#instructions-on-how-to-create-role-based-entitlement-templates). ## Automated Group Membership Reports for Managers If you would like to check whether or not a team-member is a member of a Slack or a G-Suite group, you can view the following automated group membership reports: [G-Suite Group Membership Reports](https://gitlab.com/gitlab-com/security-tools/report-gsuite-group-members) [Slack Group Membership Reports](https://gitlab.com/gitlab-com/security-tools/report-slack-group-members) ### How to contact us, or escalate priority issues outside of standard hours Team members contact details can be found in Slack profiles. Slack is not ideal for managing priorities of incoming issues, so we ask that all such requests create an issue at [IT Ops Issues](https://gitlab.com/gitlab-com/business-ops/itops/issue-tracker/issues) and we will triage and address them as soon as we can. All issues created in the queue are public by default. Privileged or private communications should be sent to itops@ where all new issues are private by default, visible only to the reporter and appropriate team members. Screenshots and videos are very helpful when experiencing an issue, especially if there is an error message. ## Laptops ### Laptop Ordering Process The laptop ordering process starts as soon as an offer is accepted by a candidate and the initial Welcome email is sent by the Candidate Experience Specialist. This email will include a link to the Notebook Order Form where the new team member will state their intent for obtaining or ordering hardware. Team members that live in these countries can be serviced via the IT Laptop Ordering Process: USA, Canada, Japan, Mexico, all of the EU, Russia, Thailand, China, Philippines, Australia, and New Zealand. Please note that we are adding supported countries to this list as we discover our ability to order in them. You can test this by going to order a Macbook Pro (or Dell) from the regional Apple store, and seeing if they let you customize a build or alternately refer you to local retailers. If the latter, see below. Team members that do not live in these countries will need to procure their own laptop and submit for reimbursement. If the team member desires financial assistance to purchase the hardware, the Company will advance the funds to help facilitate the purchase (see Exception Processes below). ### Key Performance Indicators KPI 99% of laptops will arrive prior to start date or 21 days from the date of order. ### Exception Processes If you are in a region where we are not able to have a laptop delivered, and you need to request funds be advanced in order for a local purchase to take place; Obtain two quotes from local retailers (online or physical). Email your manager with those quotes attached, requesting the funds advance and detailing the reason why (geo region, unable to have laptop delivered). Your manager will then forward their approval to Accounting for final approval and dispensation. Should a laptop not be available to a new GitLab team-member upon their start date, but is pending, interim options include: - Using personal non-windows hardware (mac, linux, mobile) - Renting and expensing non-windows hardware - Purchasing and expensing (or returning) a Chromebook If in the rare case that your laptop arrives damaged or unusuable prior to your start date, please reach out to your Candidate Experience Specialist and CC itops@gitlab.com for next steps on an immediate replacement or repair. ### Laptop Configurations GitLab approves the use of Apple and Linux operating systems, Windows is prohibited. The prohibition on Microsoft Windows is for the following reasons: - While there has been security issues with all operating systems, due to the popularity of the Windows operating system it is the main platform targeted by attackers with spyware, viruses, and ransomware. - macOS is pre-loaded onto Apple hardware. Linux is free. To take advantage of the features in Windows as a business, GitLab would have to purchase Windows Pro licensing as these business features are not available on Windows Home Edition. As many purchases of laptops have occurred with employees making the purchases and then being reimbursed by GitLab, a remote employee would typically be making a purchase of a laptop pre-loaded with Windows Home Edition. - Windows Home Edition is notoriously [hard to secure](https://www.markloveless.net/blog/2019/1/15/dealing-with-windows-10). Further information on GitLab authorized operating systems, versions, and exception process is available on the [Approved OS's](/handbook/security/approved_os.html) page. The operating system choices have obviously affected the hardware selection process. Apple hardware is the common choice for most GitLab team-members, but if you are comfortable using and self-supporting yourself with Linux (Ubuntu usually) you may also choose from the Dell builds below. NOTE: GitLab does have a corporate discount with Apple. In Apple Retail Stores, you can bring a paystub or invoice along with a photo ID to get the discount. You can also call the [Small Business line](https://www.apple.com/retail/business/) to learn more about ordering online. Discounts vary depending on the purchase, but it can range anywhere from 2-10%. ##### Apple Hardware - Everyone Else (Macbook Pro) - [13” / 256gig SSD / 16gigs RAM / Quad-Core i5 CPU](https://www.apple.com/shop/buy-mac/macbook-pro/13-inch-space-gray-256gb-2.4ghz-quad-core-processor-with-turbo-boost-up-to-4.1ghz#) - For Engineers, Support Engineers, Data Analysts, Technical Marketing Managers, Product Designers, UX Managers, Product Managers, Technical Writers, Digital Production (Macbook Pro) - [16” / 512gig SSD / 32gig RAM / i9 or i7 CPU](https://www.apple.com/shop/buy-mac/macbook-pro?product=MVVJ2LL/A&step=config) Linux Hardware **Please only request a Linux laptop if you have experience with this operating system and its requirements, all others please select a Macbook** - For Engineers, Support Engineers, Data Analysts, Technical Marketing Managers, Product Designers, UX Managers, Product Managers, Technical Writers, Digital Production (Dell/Linux) - [15” / 512gig SSD / 32gig RAM / i9 or i7 CPU ](https://www.dell.com/en-us/work/shop/cty/pdp/spd/precision-15-5530-laptop/xctop5530hwus) - Everyone Else (Dell/Linux) - [13” / 256gig SSD / 16gigs RAM / Quad-Core i5 CPU](https://www.dell.com/ng/business/p/latitude-13-7300-laptop/pd) We strongly encourage Macs, but we do allow Linux if you are capable of self-support and updates. For Linux laptops, we recommend purchasing a Dell computer pre-loaded with Ubuntu Linux (to save money by not purchasing a Windows license). The reasons for using Dell for a Linux laptop are as follows: - There are several manufacturers of laptop systems that offer Linux, but Dell is the only major manufacturer that has done so for years, and it has already worked out shipping issues for all of the countries where GitLab employees live. - As we move forward with Zero Trust networking solutions, we need to have a stable and unified platform for deployment of software components in the GitLab environment. Standardization on a single platform for Linux simplifies this. - Ubuntu 18.04 LTS is the preferred platform due to both stability and an extremely fast patch cycle - important for security patches. - There are opportunities for corporate discounts in the future if we can concentrate purchases from a single vendor. - Dell is a [certified Ubuntu vendor](https://certification.ubuntu.com/desktop/models?query=&category=Desktop&category=Laptop&level=&release=18.04+LTS&vendors=Dell) with plenty of laptop choices available. They even have their own Ubuntu OEM release of Ubuntu they maintain, and as a result of their effort, the standard Ubuntu 18.04 LTS image natively supports Dell hardware and even firmware updates. - To date, all of Dell's security issues have involved their use of Windows, not their hardware. ** NOTE : This model is not the [standard configuration of Macbook pro](https://www.apple.com/macbook-pro/specs/). Please make sure you order this model minimum 7days, based on your locality, prior to your desired date to receive. \*** NOTE : for this model, it is suggested to also purchase and expense (or request in your initial laptop order) an inexpensive webcam. The built in webcam looks straight up your nose. Also note that [1Password](/handbook/security/#1password-guide) does not yet have a native client for Linux, but [there is a browser extension](https://support.1password.com/getting-started-1password-x/). Max price: **the price of the equivalent Mac laptop** Laptops are purchased by IT Ops when a team-member comes on board; the team-member will be sent a form to fill out for ordering. ### Laptop Vendor Selection Criteria When recommending or approving end user device vendors for team member access, the Security Team tries to balance privacy, security, and compliance to ensure a solid choice for accessing GitLab data. Our current recommendations include Apple MacBook Pro running macOS and Dell Precision running Linux. By its very nature, GitLab has historically been very open as a company, starting as open source and migrating from a group of coders with their own laptops to an organization that needs to protect not just their own corporate data but customer data as well. Having developed a Data Classification Policy and currently implementing Zero Trust, we've had to make adjustments in laptop recommendations. Our laptop vendor selection criteria is as follows: ##### Team member need The main needs center around processing power and the operating system support for required workloads. Most modern systems meet the processing power needs of our team members. Apple macOS and Dell Linux distributions meet the operating system needs. ##### Security needs GitLab needs the ability to ensure a secure and stable platform. From an operating system perspective, macOS and Linux meet the basic needs. The Security team has found a slight advantage in Ubuntu as a Linux distribution due to their rapid response time when it comes to patching security flaws, and recommend this distribution. This is not at the exclusion of all other distributions, but this is the one we recommend. In the case of Microsoft Windows, as we previously stated there are a number of reasons to restrict access from a security perspective. Historically, the operating system has had its share of security flaws and is a frequent target of various forms of malware. In fact Windows is responsible for the major share of all malware, including such items as ransomware. From a hardware point of view, we have to examine security issues such as supply chain attacks. Some vendors have had these issues involving the operating system, although in many cases it has been directly related to Microsoft Windows. However, if a vendor has had potential supply chain issues involving firmware or hardware, we will consider other vendors first. ##### Compliance needs To meet compliance needs for the various certifications, programs, and industry regulations, we have to meet criteria including the ability to restrict access to sensitive data to company-issued laptops running company-monitored software. In many cases we need to be able to prove this via auditing, including outside auditors. Using one vendor for macOS (Apple by default) and one for Linux. A part of this process will include ensuring systems are patched, and in the Linux case we want to ensure firmware patches are applied. Very few hardware vendors not only supply Linux as an operating system but also provide a way to apply patches - including security patches - to firmware via the normal Linux patch process. There is no specific example for using one brand over another from a compliance perspective. That being said, there are customers we wish to sell GitLab products that have specific requirements internally, and to align ourselves with those requirements can be not only a positive sign we understand the customer space, but give us a competitive advantage. For example, since there is a [strong push](/solutions/public-sector/) to sell to agencies within the US Government, we will already face restrictions such as support from only US-citizen GitLab team members while on US soil. As these agencies also have access to classified (non-public) reports on such things as computer vendors, one only has to note which laptops are approved for purchase. For that reason, we will restrict our vendor list to vendors that are currently approved by the various organizations issuing those certifications and programs we are trying to be compliant with. This simplifies the ability to support those customers which may impose restrictions on team members working in support roles for that customer solely based upon the hardware they are using. In other words, we eliminate this possibility of becoming a situation to be managed. ##### Logistics needs To be able to use a laptop vendor, we have to be able to purchase and ship hardware to our team members regardless of where they live. Therefore the vendor should be able to handle most if not all shipping requirements to all team members. ### Laptop Refresh Team member Laptops can be refreshed after 3 years (of use!) without question. If you feel you need a new laptop to be effective at your job before then, reach out to IT and your manager. Replacement laptops for broken GitLab laptops can be purchased as needed by [creating an issue](https://gitlab.com/gitlab-com/business-ops/itops/issue-tracker/issues/new?issue) in the IT Ops issue tracker project and using the `repair/replacement` template. Laptops ordered as part of the refresh program use the same template. This process can also be followed for laptops that are not broken but old enough that you are having trouble completing your work. Please refer to the [spirit of spending company money](/handbook/spending-company-money/) when deciding whether or not it is appropriate to replace your functioning laptop. Everyone's needs are different so it is hard to set a clear timeline of when computer upgrades are necessary for all team-members, but team-members become eligible for an updated laptop after 3 years. If you qualify/complete a laptop refresh, please also refer to our [Laptop Buy back Policy](/handbook/business-ops/it-ops-team/#laptop-buy-back-policy) below. Many team members can use their company issued laptop until it breaks. If your productivity is suffering, you can request a new laptop. The typical expected timeframe for this is about three years, but it can depend on your usage and specific laptop. Laptops paid for by the company are property of GitLab and need to be reported with serial numbers, make, model, screen size and processor to IT Ops by adding it to this form: [GitLab laptop information](https://forms.gle/czgiZAT2D1Rxt7Dw6) for proper [asset tracking](/handbook/finance/accounting/#asset-tracking). Since these items are company property, you do not need to buy insurance for them unless it is company policy to do so (for example, at the moment we do not purchase Apple Care), but you do need to report any loss or damage to IT Ops as soon as it occurs. Links in the list below are to sample items, other options can be considered. ### Laptop Trade-In If you have a laptop that is older than 3 years and you intend to refresh it with a new one, we advise that you trade it in for credit towards a new laptop. There are two ways this can be done: 1. [Apple's trade-in program](https://www.apple.com/shop/trade-in). 2. [SellYourMac](https://www.sellyourmac.com/) Note that Apple's trade-in program requires sending in the laptop within 14 days of requesting a trade-in. Please also make sure the credit is applied towards a new laptop, as the intent of this policy is to save the company money and to encourage recycling. Please check with the IT Ops department if you have any questions. ### Laptop Repair If your laptop is broken and needs to be repaired you can take it into an Apple repair store. If the repair is not going to be too expensive (more than $1000 dollars USD), go ahead and repair and expense. If the repair is going to take longer than a day then you need to make sure you have a back up laptop to work on that is non-Windows. You can open an issue in [ITOPS ](https://gitlab.com/gitlab-com/business-ops/itops/issue-tracker/issues/new?issuable_template=Laptop%20Repair) to document the repair and get your managers approval. If, however, the repair is going to be expensive and take weeks to fix and you have no back up laptop, your best option is to replace the laptop. In this case please open [an issue to replace](https://gitlab.com/gitlab-com/business-ops/itops/issue-tracker/issues/new?issuable_template=Laptop%20Replacement). Then please follow the guidelines in the template and once you receive the new laptop we can have the old one sent off to our reseller. ### Configuring New Laptops New laptops should be configured with security in mind. Please refer to [security best practices](/handbook/security/#best-practices) when configuring new laptops. **All team-members must provide proof of whole-disk encryption within the new laptop order issue.** Certain circumstances (world region and availability of hardware) might require the self installation of Linux on a Dell that was shipped with OEM Windows. Please make sure you follow any needed requirements when self installing and open an issue with IT-Ops for verification. For laptops shipped with OEM Windows you may want to make a full drive backup (e.g. by using open source utility [Clonezilla](https://clonezilla.org/)) to the external drive before installing Linux. That way you could restore your laptop to the original state at any time. It will make the RMA process much easier in case you need it. ### Laptop Buy back Policy Team members can choose to refresh their laptop, no questions asked, after 3 years of use (not necessarily 3 years of employment if a used laptop was issued at the time of onboarding). Team members have the option to buy back their existing laptops either when it gets refreshed for a new one, or when the team member is offboarding. If the team member has completed 1 calendar year or more at GitLab at the time of laptop refresh or offboarding, they can opt to keep their laptop at no cost. If the team member hasn't completed 1 calendar year at GitLab at that time, they have the option to purchase their laptop for current market value. IT Ops will email the team member asking if they would like to send back or purchase their laptops. If purchasing, our Business Ops Director will approve, and we will send the employee an email with the determined value. Then, if the employee decides to move forward with purchasing, our accounting department will reach out with payment information. If a team member decides to retain their laptop, they are required to wipe the machine and re-install the base operating system, and remove any and all software and configurations that were supplied by GitLab. Evidence or a declaration that the device has been wiped must be supplied to GitLab within 2 weeks of the end of employment. If GitLab discovers that a device has not been wiped according to policy, GitLab may act to enforce a remote wipe without notice. If team members opt not to keep or purchase their existing laptops, they can return them to GitLab. See the [returning old/offboarded laptops](#returning-oldoffboarded-laptops) section below for details. ### Returning Old/OffBoarded Laptops Part of the IT Ops replacement laptop process is providing each team-member with instructions about how to return their old laptop (whether outdated or broken). All laptops must be returned **within 2 weeks of receiving the replacement laptop**, so please prioritize transferring information between laptops within this timeframe. If an offboarded employee decides not to purchase, then we will have them ship to our 3rd party vendor that handles sell backs, SellYourMac. SYM will send them a shipping label, and in the US, a shipping box as well. All team-member laptops must be securely erased before being returned. This not only protects the company, but also protects you since it is possible for personal information to exist on these machines. Reformatting a computer is not sufficient in these cases because it is possible for sensitive data to be recovered after reinstalling an operating system. ## Other Resources ### Okta In an effort to secure access to systems, GitLab is utilizing Okta. The key goals are: - We can use Okta to enable Zero-Trust based authentication controls upon our assets, so that we can allow authorized connections to key assets with a greater degree of certainty. - We can better manage the login process to the 80+ and growing cloud applications that we use within our tech stack. - We can better manage the Provisioning and De-provisioning process for our users to access these application, by use of automation and integration into our HRIS system. - We can make Trust and Risk based decisions on authentication requirements to key assets, and adapt these to ensure a consistent user experience. To read more about Okta, please visit the [Okta](/handbook/business-ops/okta/) page of the handbook. ### Full Disk Encryption To provide proof of Full Disk Encryption, please do the following depending on the system you are running. - Apple : Take a screenshot showing both the confirmation of enabled Full Disk Encryption as well as the info showing your serial number. Both pieces of information can be found by clicking on the Apple icon in the top left corner of your screen. For proof of disk encryption, choose `System Preferences -> Security & Privacy`, and then choose the `FileVault` tab near the top of the window. For your serial number, choose the `About This Mac` option. Please get both pieces of information in a single screenshot. - Linux : Take a screenshot showing the output of ``sudo dmsetup ls && sudo dmidecode -s system-serial-number && cat /etc/fstab`` ### Fleet Intelligence & Remote Lock/Wipe (DriveStrike) GitLab has a large and ever-growing fleet of laptops, which IT Operations is responsible for maintaining. In order to do this and combined with our Zero Trust security policies and various Compliance needs, there must be some measure of intelligence and reporting in place. To accomplish this goal we are utilizing DriveStrike as a light-touch mechanism to obtain only the essential information required. DriveStrike is available for all operating systems, and also meets security needs for remote lock or wipe in emergencies. The initial stage of this rollout will take the form of an Open (Voluntary) Sign-Up for DriveStrike for all GitLab team members. The self-serve enrollment form can be found here : [https://docs.google.com/forms/...](https://docs.google.com/forms/d/e/1FAIpQLSejQkU5gTgxZCRGawB3B7XIC0LfZr6mt7TqsoRf17Tx5L8q7Q/viewform) Questions and discussion for the Open Beta rollout can be found here: [https://gitlab.com/gitlab-com/business-ops/itops/issue-tracker/issues/251](https://gitlab.com/gitlab-com/business-ops/itops/issue-tracker/issues/251) DriveStrike installs for GitLab will NOT have the ability to execute remote commands. DriveStrike installs for GitLab WILL have the ability to remotely lock or wipe hardware, for use in emergency or offboarding situations. DriveStrike may be installed on non-GitLab hardware, opting-in to the same data collection and security. DriveStrike collects the following information for GitLab: - Usernames & Accounts present on the machine - Machine Name, Make, Model, MAC, Serial # - Hardware Specifications (CPU, RAM, HDD + % Used) - Firewall status - OS Version & Patch/Update status - Disk encryption status - Battery Health This light-touch reporting allows us to meet business and compliance needs, while maintaining the privacy of the GitLab team member. This will remain a top consideration throughout the process. ### Fleet Intelligence (Fleetsmith) FleetSmith evaluation as a reporting tool has been deprecated in favor of DriveStrike listed above. If you have FleetSmith installed, please follow the instructions above to enroll in the DriveStrike roll-out, while also uninstalling FleetSmith from your device. ### GitLab.com Apple ID's We require the use of an @gitlab.com Apple ID that is separate from any personal Apple ID's you may have. Some of these reasons include: - Backups, keychains and documents are all considered sensitive information, and should not be stored in personal services. - 2FA for remote lock, wipe, or account resets are common methods of account compromises, and ensuring the use of GitLab.com email addresses also ensures we are in control of that aspect of multi-factor authentication. - Keeping a strong separation between work and personal accounts will help prevent the accidental leak of information from one to the other, in either direction. Defense in depth, in part, means you make a best effort to be secure at each layer.