Server maintenance

A comprehensive guide to setting up Linux servers for security, stability, observability & maintainability.

Provisioning (creating)

Provisioning (creating)

Provisioning (creating)

Via Laravel Forge

When provisioning a server via Laravel Forge, make sure to do the following:

  1. Give it a name in the following syntax: '[clientname]-[environment]'. For example: silo-acceptance.
  2. Select DigitalOcean as the hosting provider.
  3. Make sure to select a suitable Laravel Forge Circle (user group) to add it to.
  4. Carefully consider the best region to host the server.
  5. Choose a VPC (Virtual Private Cloud; a private network) suited to the client and environment, which should have the syntax 'vpc-[clientname]-[environment]'. For example: vpc-silo-acceptance. If no suitable one exists, navigate to DigitalOcean to create it first and restart the server creation process in Laravel Forge.
  6. Choose appropriate specs for the server. Upsizing it later is easier than downsizing.
  7. Consider which dependencies (e.g. PHP / Database server) should be installed, and disable ones that are not needed.
  8. Enable the weekly DigitalOcean backups options.
  9. Create a Server item in 1Password, in the appropriate safe. Make sure to name it '[servername] DO' (DO indicating DigitalOcean) and to include the following details: IP-address and all login credentials presented by Laravel Forge during the creation process (e.g., the password for Linux user forge, the database server password, etc.)

After creation:

  1. Navigate to the Warpgate (https://warpgate.pnct.nl) and add the new server there (see Configuring a server in the Warpgate).
  2. Navigate to DigitalOcean and configure the firewall (see Configuring firewalls for DigitalOcean servers).
  3. Navigate to New Relic and connect the new server there (see Configuring a server in NewRelic).

Via DigitalOcean

TBA

Configuring

Configuring

Configuring a server in the Warpgate

For connecting with SSH to servers that we host, we use our self-hosted SSH bastion Warpgate. That allows us to manage the active employees and their SSH pubkeys, configure what servers they have access to, and keep logs of SSH sessions.

To configure (add) a server to the Warpgate, follow these steps:

  1. Connect to the Pionect VPN and navigate to warpgate.pnct.nl.
  2. Log in as the admin user (not with Google).
  3. You will be presented with a list of servers. At the top, press the option 'Manage Warpgate'. This will bring you to the management portal.
  4. In the header navigation, click 'Config' and then 'Targets'. You will now see a list of all the servers configured with the Warpgate.
  5. Press 'Add target', fill in the server name (as it appears on DigitalOcean/the hosting provider) as the target name, and press 'Create target'. You will now see the configuration page for the new target.
  6. Configure the following:
    1. Target host: This should be either the static IP of the server (preferred) or the dynamic IPV4 address.
    2. Username: If the server was provisioned with Laravel Forge, this must be 'forge'. Otherwise, it may be 'root'.
    3. Allow access for roles: Always select 'warpgate:admin' and the correct specific role (a.k.a. group) for the server. It should not be added to both the relevant 'Level 1' and 'Level 2' groups, only one of these must be chosen. If the relevant roles are not available, please first create them under the Roles menu before continuing. Please keep in mind the principle of least privilege when configuring this section.
  7. Press 'Check host key'. If this fails, please add the Warpgate's SSH pubkey to the authorized_keys list on the server before retrying. You can find this pubkey here (use the short one).
  8. Press 'Update' to persist your changes.
Configuring

Configuring firewalls for DigitalOcean servers

TBA

Configuring

Configuring a server in NewRelic

TBA