# Server maintenance

A comprehensive guide to setting up Linux servers for security, stability, observability & maintainability.

# Provisioning (creating)

# Provisioning (creating)

## Via Laravel Forge

When provisioning a server via Laravel Forge, make sure to do the following:

1. Give it a name in the following syntax: '<span style="text-decoration: underline;">\[clientname\]-\[environment\]</span>'. For example: <span style="text-decoration: underline;">silo-acceptance</span>.
2. Select DigitalOcean as the hosting provider.
3. Make sure to select a suitable Laravel Forge Circle (user group) to add it to.
4. Carefully consider the best region to host the server.
5. Choose a VPC (Virtual Private Cloud; a private network) suited to the client and environment, which should have the syntax '<span style="text-decoration: underline;">vpc-\[clientname\]-\[environment\]</span>'. For example: <span style="text-decoration: underline;">vpc-silo-acceptance</span>. If no suitable one exists, navigate to DigitalOcean to create it *first* and restart the server creation process in Laravel Forge.
6. Choose appropriate specs for the server. Upsizing it later is easier than downsizing.
7. Consider which dependencies (e.g. PHP / Database server) should be installed, and disable ones that are not needed.
8. Enable the weekly DigitalOcean backups options.
9. Create a Server item in 1Password, in the appropriate safe. Make sure to name it '<span style="text-decoration: underline;">\[*servername*\] *DO*</span>' (DO indicating DigitalOcean) and to include the following details: IP-address and all login credentials presented by Laravel Forge during the creation process (e.g., the password for Linux user forge, the database server password, etc.)

After creation:

1. Navigate to the Warpgate ([https://warpgate.pnct.nl](https://warpgate.pnct.nl)) and add the new server there (see [Configuring a server in the Warpgate](https://wiki.pnct.nl/books/server-maintenance/page/configuring-a-server-in-the-warpgate "Configuring a server in the Warpgate")).
2. Navigate to DigitalOcean and configure the firewall (see [Configuring firewalls for DigitalOcean servers](https://wiki.pnct.nl/books/server-maintenance/page/configuring-firewalls-for-digitalocean-servers "Configuring firewalls for DigitalOcean servers")).
3. Navigate to New Relic and connect the new server there (see [Configuring a server in NewRelic](https://wiki.pnct.nl/books/server-maintenance/page/configuring-a-server-in-newrelic "Configuring a server in NewRelic")).

## Via DigitalOcean

TBA

# Configuring

# Configuring a server in the Warpgate

For connecting with SSH to servers that we host, we use our self-hosted SSH bastion <span style="text-decoration: underline;">Warpgate</span>. That allows us to manage the active employees and their SSH pubkeys, configure what servers they have access to, and keep logs of SSH sessions.

To configure (add) a server to the Warpgate, follow these steps:

1. Connect to the Pionect VPN and navigate to [warpgate.pnct.nl](https://warpgate.pnct.nl).
2. Log in as the admin user (not with Google).
3. You will be presented with a list of servers. At the top, press the option '<span style="text-decoration: underline;">Manage Warpgate</span>'. This will bring you to the management portal.
4. In the header navigation, click '<span style="text-decoration: underline;">Config</span>' and then '<span style="text-decoration: underline;">Targets</span>'. You will now see a list of all the servers configured with the Warpgate.
5. Press '<span style="text-decoration: underline;">Add target</span>', fill in the server name (as it appears on DigitalOcean/the hosting provider) as the target name, and press '<span style="text-decoration: underline;">Create target</span>'. You will now see the configuration page for the new target.
6. Configure the following: 
    1. <span style="text-decoration: underline;">Target host</span>: This should be either the static IP of the server (preferred) or the dynamic IPV4 address.
    2. <span style="text-decoration: underline;">Username</span>: If the server was provisioned with Laravel Forge, this *must* be '<span style="text-decoration: underline;">forge</span>'. Otherwise, it may be '<span style="text-decoration: underline;">root</span>'.
    3. <span style="text-decoration: underline;">Allow access for roles</span>: Always select '<span style="text-decoration: underline;">warpgate:admin</span>' and the correct specific role (a.k.a. group) for the server. It should not be added to both the relevant 'Level 1' and 'Level 2' groups, only one of these must be chosen. If the relevant roles are not available, please first create them under the Roles menu before continuing. Please keep in mind the principle of least privilege when configuring this section.
7. Press '<span style="text-decoration: underline;">Check host key</span>'. If this fails, please add the Warpgate's SSH pubkey to the <span style="text-decoration: underline;">authorized\_keys</span> list on the server before retrying. You can find this pubkey [here](https://warpgate.pnct.nl/@warpgate/admin#/config/ssh) (use the short one).
8. Press '<span style="text-decoration: underline;">Update</span>' to persist your changes.

# Configuring firewalls for DigitalOcean servers

TBA

# Configuring a server in NewRelic

TBA