https://www.edrandall.uk/lab-notes/ Recent content in Lab Notes on Ed Randall https://www.edrandall.uk/social-card.png https://www.edrandall.uk/social-card.png Hugo -- gohugo.io en-GB Sun, 10 Nov 2024 09:07:39 +0000 https://www.edrandall.uk/lab-notes/win-serv/ Sun, 10 Nov 2024 09:07:39 +0000 https://www.edrandall.uk/lab-notes/win-serv/ AWS Windows Server Deployment with Terraform This is a simple Terraform configuration for deploying a Windows Server instance on AWS—a useful setup if you occasionally need a quick Windows environment for testing, development, or troubleshooting. It provisions a basic infrastructure including a VPC, subnet, and an RDP-accessible Windows Server instance. Access is restricted to your own IP address to provide a reasonable layer of security without over-complicating things. Code The code can be found in this github repository. https://www.edrandall.uk/lab-notes/database/ Sun, 08 Sep 2024 10:00:00 +0000 https://www.edrandall.uk/lab-notes/database/ As I prepared for the CompTIA DataSys+ exam, I gathered a series of notes and observations that helped me deepen my understanding of the database concepts which are covered in the exam's syllabus. In this post, I’m sharing some of those details. Some of the topics discussed extend beyond the scope of the exam, and not everything included in the exam's syllabus is documented. This post also contains details of the test database I designed for hands-on practice. https://www.edrandall.uk/lab-notes/time-machine-docker/ Tue, 16 Apr 2024 10:00:47 +0000 https://www.edrandall.uk/lab-notes/time-machine-docker/ This post explains how I run samba and avahi-daemon inside a docker container to act as a time-machine server for all the Mac computers on my home network. Dockerfile I used ubuntu:focal as a base image and created the following Dockerfile: FROM ubuntu:focal #Install samba and avahi-daemon inside the image RUN apt update && apt install samba avahi-daemon -y #Expose the ports which are needed for samba and avahi-daemon EXPOSE 137/udp 138/udp 139 445 5353/udp #Copy config files into the image COPY smb. https://www.edrandall.uk/lab-notes/wireguard-ec2-vpn/ Thu, 28 Dec 2023 00:00:00 +0000 https://www.edrandall.uk/lab-notes/wireguard-ec2-vpn/ This quick terraform project sets up an EC2 instance, configures wireguard, generates the required keys (including a PSK) and provides the wireguard “Client” configuration. The code can be found here WARNING: Cloud network egress/throughput will incur costs which can vary from region to region. Purpose This terraform code (and associated template file containing a bash script) will create an EC2 instance which is running wireguard and can be used as a VPN endpoint. https://www.edrandall.uk/lab-notes/kubernetes-on-ec2/ Sun, 27 Aug 2023 09:00:00 +0000 https://www.edrandall.uk/lab-notes/kubernetes-on-ec2/ Github Repo All the code required can be found in the github repo: github.com/edrandall-dev/kubernetes-on-ec2 Introduction Last year, when I was learning Kubernetes, I wanted to create my own cluster on AWS using EC2 instances. The idea behind this was go through the installation of Kubernetes from start to finish, learning everything I needed to along the way. I’ve also used Amazon’s Elastic Kubernetes Service (EKS) to deploy Neo4j using the official Neo4j Helm Charts. https://www.edrandall.uk/lab-notes/cloudflare-tunnels/ Sat, 15 Jul 2023 00:00:00 +0000 https://www.edrandall.uk/lab-notes/cloudflare-tunnels/ Setting up your own VPN can be complex In this post, I went through how I’ve setup Wireguard to be able to connect back to my home network when on the road. When it comes to choosing a VPN, Wireguard stands out for me in terms of speed, security and ease of configuration. However, there are a few complexities associated with setting up any VPN to securely connect back to a home network. https://www.edrandall.uk/lab-notes/kubernetes-notes/ Tue, 04 Jul 2023 08:00:00 +0000 https://www.edrandall.uk/lab-notes/kubernetes-notes/ Overview This post covers 3 major topics: Kubernetes Lab Environment: The steps that I took to setup a Kubernetes cluster from scratch, in a self-hosted virtualised environment using Oracle Virtualbox and Hashicorp Vagrant Kubernetes Notes, Commands and Manifests: A collection of commands which I noted down as part of my preparation for the Certified Kubernetes Administrator exam Information Sources: A list of links to the main sources of information which I used when creating this environment and learning Kubernetes Kubernetes Lab Environment The VMs which formed the Kubernetes cluster were deployed on a Dell Precision workstation with plenty of CPU and RAM resources: https://www.edrandall.uk/lab-notes/helm/ Wed, 01 Mar 2023 09:00:00 +0000 https://www.edrandall.uk/lab-notes/helm/ In this post, I will go over the steps needed to provision a Kubernetes cluster in AWS, and deploy a Neo4j cluster using Neo4j’s helm charts. This post takes steps from the official Neo4j Kubernetes Documentation Prerequisites An AWS Account Configured AWS Command Line Interface An SSH Key named id_rsa.pub. If you do not have one, you can generate it by running: ssh-keygen -t rsa -C "your-name@example.com" Setting up a Kubernetes Cluster in EKS Installation of tools and applications In order to create a Kubernetes (EKS) cluster in AWS, you will need to download the following applications: kubectl https://www.edrandall.uk/lab-notes/neo4j-aws-quickstart/ Fri, 24 Feb 2023 10:00:00 +0000 https://www.edrandall.uk/lab-notes/neo4j-aws-quickstart/ My post on the neo4j blog explains how to get up and running with Neo4j using the AWS partner solution. AWS Partner Solutions (formerly known as AWS Quick Start) are pre-configured reference deployments that work seamlessly with Amazon Web Services (AWS) and follow AWS’s recommended design practices. They provide customers with a variety of deployment options for managing their workloads on AWS. Neo4j’s AWS Partner Solution for AWS is a quick and easy way to deploy a fully-managed Neo4j cluster on the AWS platform. https://www.edrandall.uk/lab-notes/hugo/ Mon, 30 Jan 2023 22:10:00 +0000 https://www.edrandall.uk/lab-notes/hugo/ This post describes how I got this website up and running using Hugo, on cloudflare pages. Install Hugo To install hugo, follow the instructions on the hugo website. If you’re on a mac (and have homebrew installed) the simplest way is: brew install hugo With hugo installed the next stage is to create the new site, which is done locally. The following commands will create the framework for the new website. https://www.edrandall.uk/lab-notes/aws-private-lb/ Thu, 26 Jan 2023 22:43:00 +0000 https://www.edrandall.uk/lab-notes/aws-private-lb/ Recently, I had to re-visit an old architectural challenge in AWS that I came across a couple of years ago but couldn’t quite remember how to resolve: EC2 instances which have no public IP address (residing in a private subnet) which can accept requests from (and serve responses back to) the internet. The motivation for this particular topology is security focused: The EC2 instances cannot be directly addressed via the internet due to the absence of a public IP, but can serve (web, in this example) content over the internet using the load balancer. https://www.edrandall.uk/lab-notes/tf-modules-vars/ Fri, 06 Jan 2023 11:40:00 +0000 https://www.edrandall.uk/lab-notes/tf-modules-vars/ In this post, I outline the steps needed to take an input from a terraform parent (or calling) module and pass it to a config file on a target EC2 instance. What are Terraform Modules? Using modules in terraform helps to organise and reuse code, allowing a module to be defined once and used many times. For example, a development team may want to quickly create and destroy a number of EC2 instances. https://www.edrandall.uk/lab-notes/terraform-subnets/ Tue, 20 Dec 2022 17:25:00 +0000 https://www.edrandall.uk/lab-notes/terraform-subnets/ The following code snippets show how to create three private subnets in AWS using terraform with ‘count’, ‘join’ and ’element’: To start with the following variables were declared and set: terraform.tfvars base_cidr_block = "192.168.0.0/16" node_count = 3 availability_zones = ["a", "b", "c"] variables.tf variable "base_cidr_block" { description = "The base of the address range to be used by the VPC and corresponding Subnets" } variable "availability_zones" { description = "A list containing 3 AZs" type = list(string) } variable "node_count" { description = "The number of ec2 instances to deploy" type = number } Three subnets are then created using a single resource block. https://www.edrandall.uk/lab-notes/wireguard/ Tue, 20 Dec 2022 10:25:00 +0000 https://www.edrandall.uk/lab-notes/wireguard/ I wanted to use create a VPN back to my home network which I could use on the road. The idea was that I could interact with resources on my home network but also have a secure tunnel which I could use if I was needed to use an untrusted WiFi hotspot when in hotels, coffee shops etc. I had previously implemented OpenVPN to do this, with the VPN server running on a small linux machine inside my home network. https://www.edrandall.uk/lab-notes/4g-backup/ Mon, 19 Dec 2022 14:23:47 +0000 https://www.edrandall.uk/lab-notes/4g-backup/ Like most people, my domestic ISP goes down sometimes. I work from home most of the time and the most common solution when this problem occurs is to ’tether’ a phone to use it’s 4G connection. This is OK, but a solution which allowed my entire home network to switch over to a backup internet connection would be much better… As well as my laptop maintaining a connection, movies could still be streamed (backup connection bandwith permitting) and IoT devices would keep working - you get the general idea. https://www.edrandall.uk/lab-notes/neo4j-docker/ Fri, 11 Mar 2022 10:00:47 +0000 https://www.edrandall.uk/lab-notes/neo4j-docker/ This is a very quick script to get a single instance of neo4j running in a docker container. The username and password is set using the NEO4J_AUTH environment variable. #!/bin/bash # # Script: neo4j-docker.sh # Purpose: install neo4j container image after some simple environmentment prep # DOCKER=$(which docker) [ $? == 0 ] || { echo "ERROR: Check if docker is installed (and in your PATH)" ; exit 1; } DATA_DIR="$HOME/docker/neo4j-volumes/data" LOG_DIR="$HOME/docker/neo4j-volumes/logs" IMPORT_DIR="$HOME/docker/neo4j-volumes/import" PLUGINS_DIR="$HOME/docker/neo4j-volumes/plugins" RUNNING_CONTAINER=$($DOCKER ps -a | grep "neo4j:latest" | awk {'print $1'}) [ -z "$RUNNING_CONTAINER" ] || docker rm -f $RUNNING_CONTAINER [ -d $DATA_DIR ] && rm -rf $DATA_DIR/* || mkdir $DATA_DIR [ -d $LOG_DIR ] && rm -rf $DATA_DIR/* || mkdir $LOG_DIR [ -d $IMPORT_DIR ] && rm -rf $DATA_DIR/* || mkdir $IMPORT_DIR [ -d $PLUGINS_DIR ] && rm -rf $DATA_DIR/* || mkdir $PLUGINS_DIR $DOCKER run \ --name testneo4j \ -p7474:7474 -p7687:7687 \ -d \ -v $DATA_DIR:/data \ -v $LOG_DIR:/logs \ -v $IMPORT_DIR:/var/lib/neo4j/import \ -v $PLUGINS_DIR:/plugins \ --env NEO4J_AUTH=neo4j/test1234 \ neo4j:latest https://www.edrandall.uk/lab-notes/restoring-webapps/ Fri, 16 May 2014 12:00:00 +0000 https://www.edrandall.uk/lab-notes/restoring-webapps/ Restoring Web Applications for edrandall.co.uk Section A - Install & Configure the WebServer with SSL Support Install the required ubuntu webserver packages aptitude install apache2 php5 php5-mcrypt php5-curl Create the following webserver configuration files: * /etc/apache2/conf.d/ejr-servers.conf NameVirtualHost 192.168.2.2:443 * /etc/apache2/sites-available/restore.edrandall.co.uk ServerName restore.edrandall.co.uk <VirtualHost 192.168.2.2:443> SSLEngine On SSLCertificateFile /etc/apache2/ssl/server.crt SSLCertificateKeyFile /etc/apache2/ssl/server.key DocumentRoot /var/www/ <Directory "/var/www"> Order deny,allow Deny from all Authname "restore.edrandall.co.uk" AuthUserFile /var/www/.htpasswd AuthType Basic Require valid-user Allow from 192.168.2. Satisfy Any </Directory> </VirtualHost> * /etc/apache2/ports. https://www.edrandall.uk/lab-notes/rhce-notes/ Fri, 29 Jun 2012 12:00:00 +0000 https://www.edrandall.uk/lab-notes/rhce-notes/ A) iptables script #!/bin/bash ipt=$(which iptables) #Flush all existing rules $ipt -F #Set default policies $ipt -P INPUT DROP $ipt -P OUTPUT ACCEPT $ipt -P FORWARD DROP $ipt -A INPUT -i lo -j ACCEPT #Allow stuff back in that we’ve sent out $ipt -A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT #Allow inbound SSH to this box $ipt -A INPUT -p tcp --dport 22 --source 172.16.0.0/16 -j ACCEPT #Save the rules /sbin/service iptables save #Show the rules $ipt --list -n -v Test (from desktop machine): https://www.edrandall.uk/lab-notes/luks-encyption-redhat/ Tue, 21 Feb 2012 12:00:00 +0000 https://www.edrandall.uk/lab-notes/luks-encyption-redhat/ Creating an Encrypted Disk Partition Created a new logical partition (/dev/vda5) which is 500MB in size After a reboot, check the partition is there and looks okay: fdisk -cul /dev/vda Prepare the partition for encryption (enter YES and passphrase when prompted): cryptsetup luksFormat /dev/vda5 Unlock the encrypted partition cryptsetup luksOpen /dev/vda5 encpartition Have a look in the /dev/mapper directory, there’s an entry for /dev/mapper/encrypted partition: lrwxrwxrwx. 1 root root 7 Feb 21 12:24 /dev/mapper/encpartition -> . https://www.edrandall.uk/lab-notes/cobbler/ Mon, 09 Jan 2012 12:00:00 +0000 https://www.edrandall.uk/lab-notes/cobbler/ Download the epel packages from: http://fedoraproject.org/wiki/EPEL Edit the files: /etc/yum.repos.d/epel-repo and /etc/yum.repos.d/epel-testing.repo Uncomment the following line: #baseurl=http:// Comment the following line (Otherwise yum won’t find the epel packages and won’t work properly.): mirrorlist=http:// Install cobbler: yum install cobbler policycoreutils-python-2.0.83-19.18.el6.i686 Install mod_python for apache yum install mod_python Install dhcp yum install dhcp Install syslinux and syslinux-tftpboot yum install syslinux syslinux-tftpboot Cobbler and SELinux don’t play nice together. Whilst it is possible to configure cobbler to work with selinux, it is easier to disable it altogether. https://www.edrandall.uk/lab-notes/multiple-files/ Thu, 03 Mar 2011 12:00:00 +0000 https://www.edrandall.uk/lab-notes/multiple-files/ I know there’s many ways to do this, but I quite like this method for a batch file rename: ls foo*.jpg | awk '{print("mv "$1" "$1)}' | sed 's/foo/bar/2' | /bin/bash If you want to do a dry run first, then just omit the pipe to bash: ls foo*.jpg | awk '{print("mv "$1" "$1)}' | sed 's/foo/bar/2' https://www.edrandall.uk/lab-notes/netcat-tar-pipe/ Tue, 01 Mar 2011 12:00:00 +0000 https://www.edrandall.uk/lab-notes/netcat-tar-pipe/ Sometimes SCP, SFTP or other methods of file transfer aren’t available. Using netcat and tar, files can be archived and sent over the network on the fly, like this: On the destination server: mkdir /destination/directory cd /destination/directory nc -l 7000 | tar x On the source server: cd /source/directory tar cf - * | nc destination-hostname 7000 https://www.edrandall.uk/lab-notes/iptables-logdrop/ Thu, 25 Nov 2010 10:07:33 +0000 https://www.edrandall.uk/lab-notes/iptables-logdrop/ Its useful to have a dedicated logfile which contains details of packets which are dropped or rejected by your iptables firewall. This will allow to you see when connection attempts are being made to your server by IP addresses which are being blocked. This can also be extremely useful when troubleshooting your iptables firewall. If you’re not up to speed with iptables, then you’ll need to find another guide to work through. https://www.edrandall.uk/lab-notes/mysql-slave/ Wed, 24 Nov 2010 12:40:00 +0000 https://www.edrandall.uk/lab-notes/mysql-slave/ On the primary DB node: Enable binary logging in my.cnf: [mysqld] log_bin=mysql-bin server-id=1 Restart MySQL /etc/init.d/mysqld restart Create a user for replication: mysql> GRANT REPLICATION SLAVE ON *.* TO 'user'@'1.2.3.4' IDENTIFIED BY 'slavepass'; Find filename of binlog: mysql> show master status; Do the mysqldump of the database you want to replicate: mysqldump -c –create-options -u root -ppassword –lock-tables databasename > dump.sql Copy the sql file over to the slave: scp dump. https://www.edrandall.uk/lab-notes/mount-disk-label/ Wed, 24 Nov 2010 12:20:00 +0000 https://www.edrandall.uk/lab-notes/mount-disk-label/ USB hard disks are often allocated different device names by Linux each time the system boots. For example: what is currently recognized as /dev/sda1 may well end up being /dev/sdb1 after a reboot. To get round this, it is possible to mount disks by label rather than device name by taking the following steps (as root): First of all, you need to see if the partition in question already has a label: https://www.edrandall.uk/lab-notes/self-signed-cert/ Wed, 24 Nov 2010 12:10:00 +0000 https://www.edrandall.uk/lab-notes/self-signed-cert/ Creating a self-signed certificate for an apache server is fairly straightforward. The following steps show you how to do it on an ubuntu / debian based system running apache: If it doesn’t already exist, make the directory for the key, csr and certificate to go in: mkdir /etc/apache2/ssl Go to that directory: cd /etc/apache2/ssl Generate the private key (Enter a passphrase when prompted, we’ll remove this later) openssl genrsa -des3 -out server. https://www.edrandall.uk/lab-notes/case-change/ Wed, 24 Nov 2010 12:00:00 +0000 https://www.edrandall.uk/lab-notes/case-change/ Here’s an easy way change the names of all files in a directory from upper to lower case (note that we are using ls -1 and not ls -l in this example): user@server:~/test$ ls -1 FIVE FOUR ONE THREE TWO user@server:~/test$ for file in `ls -1` ; do newfile=`echo $file | tr .A-Z. .a-z.` ; mv $file $newfile ; done user@server:~/test$ ls -1 five four one three two To invert the process, simply change reverse the options passed to the ‘tr’ command like this: