Node Diggity

node_logo

I’m getting fully stuck into Node.js today, writing a POC service with a number of REST endpoints, using Express. I do like the quick simplicity of setting up entire services in this way, often in just a single file, there’s always that buzz you get from the initial rapid development of any project in its early stages.

One thing I will say, though, is that I do have doubts about my own ability/knowledge when it comes to securing a Node app, as opposed to a Spring/Java service which I’m a lot more accustomed to. I need a bit more experience with Node in production before I reach a reasonable level of confidence in security matters.

In this case, however, the service I am writing is a purely public search engine wrapper, with only GET endpoints and no intrinsic vulnerabilities provided by the endpoints themselves (I have taken care of the obvious SQL injection stuff etc.). With that in mind, I don’t have any huge concerns in deploying the finished version to production.

Best crack on.

 

TRP

Advertisements

NFS (Now F**king Sorted)

victory_dance_2

I finally managed to get the NFS mount working, and this is how I did it…

NOTE: This is the initial setup before I’ve secured it up a bit, just for a POC, so don’t do this yourself without doing some extra work to secure it later…

Server Side

Installation

$ sudo apt-get update
$ sudo apt-get install nfs-kernel-server

Configuration

1. Create folder for sharing

$ cd /var
$ sudo mkdir nfs
$ sudo chmod -R 777 nfs

2. Add file

$ cd nfs
$ sudo vi test.txt

(add any old text in, and save)

3. Configure exports

$ sudo vi /etc/exports

Add the following line

/var/nfs  *(rw,sync,subtree_check,insecure)

4. Export and start service

$ sudo exportfs -a
$ sudo service nfs-kernel-server start

5. Confirm installed and running

$ ps aux | grep nfs
$ showmount -e

Client Side

1. Confirm we can see NFS mounts

$ showmount -e {server IP address}

ERROR

At this point, things weren’t going well. I could not connect to NFS, even just to list mounts. In order to diagnose this, I looked at outgoing traffic from my Mac, using:

$ sudo tcpdump host {server IP address}

This showed me that connections were certainly being attempted from the Mac, but the Ubuntu server was giving nothing back. The next step was to determine if the server firewall was rejecting requests.

Running this on the server, whilst attempting showmount -e {server IP address} again on the client confirmed that connections were being dropped:

$ sudo tail -F /var/log/kern.log

I had some over-eager rules in ufw that were blocking the NFS requests (even though an NMAP port scan showed all the required ports being open, etc.) and once I’d stopped the firewall from blocking connections I could confirm that the folder was now ready to mount.

2. Mount

$ sudo mount -o rw -t nfs {server IP address}:/var/nfs /path/to/local/folder

3. Confirm that mount was successful

$ cd /path/to/local/folder
$ ls
$ cat test.txt

4. Unmount

$ sudo umount -f /path/to/local/folder

And that is about it. Obviously, there are some places where security can immediately be tightened (eg. Don’t just have 777 permissions on the folder, and use specific IP ranges in /etc/exports instead of just *) but this should help in getting NFS up and running initially.

I may revisit this again once my knowledge has improved a bit.

 

TRP

NFS(FW)

dwight_rage

AAAAAAAaaaaaargh! Fuck’s sake! I’m having a lot of sport trying to set up an NFS mount between my Mac and work’s Linux server.

It seems to be running okay on the server side, I can see that NFS is running as a service via:

$ ps aux | grep nfs

And also see that the mount is “ready” via:

$ showmount -e

But I can’t get any kind of client connection to the server, despite ping and nmap working okay. I hate networking stuff!

I have thrown myself on the mercy of the StackExchange/ServerFault community, and await with baited breath the solution to this particular pain in my arse. Will update this post if/when I get it sorted…

TRP

Skill Up!

skill-development-training-255x224

I’ve been spending a bit of time skilling up in work, as I’m settling in and learning the ropes of the new place. As previously discussed, I’ve properly looked at Docker (finally), but I’ve also been improving my knowledge of:

  • Python
  • jQuery
  • Bash scripts
  • NFS

I think my next targets for consideration are:

  • Graph Databases
  • Solr vs. ElasticSearch
  • MapReduce

And here concludeth the most boring blog post I’ve done to date.

 

TRP

Bug Hunting

setting_snare

I’m just in the middle of a two-day bug hunt, and I’m poised to “make the kill” tonight when I get a chance to fully run the app, so I thought I would share my thoughts on two different styles of catching bugs: Stalking, and Laying Traps.

Stalking

A lot of developers I’ve worked with prefer to catch a bug red-handed by stepping through the code with a debugging tool. I compare this approach to consciously stalking your “prey”, specifically chasing it down and remaining fully focussed throughout the process.

When you find the bug, you have to be careful to capture the information you require before it (possibly) gets lost, and you must be careful not to step past it or you might end up back at square one. This can be a frustrating experience, especially if the bug is intermittent or generally hard to reproduce.

Laying Traps

My preferred method is to lay some traps. I add logging (or, more usually, highly temporary System.out.println calls) such that there is no way the bug can execute without revealing the exact pathway it took through the code. These are like traps that you can return to at your leisure and check if you’ve caught anything.

I prefer this for a couple of reasons: 1) I don’t have to be “on the ball” when the code is run, I can just pick up the output whenever I’m ready and – if I’ve done it right – it will definitely reveal how (or at least where) things are going wrong, and 2) This is the only way to reliably debug a multi-threaded application.

 

So, I’ve got my traps set for tonight. This is probably my favourite part of catching bugs – with everything set up and ready to snare the little bastard that’s been annoying me for a while. The actual fix is never as much fun.

 

TRP

Consider Yourself Dockerized!

anchorman_jump

I’ve just managed to “Dockerize” my base SpringBoot application, so I’m now able to use this as a template for future services. I’d wasted some time following various confusing tutorials, but it turned out to be very simple in the end.

Create Dockerfile

Create a new file in the root directory of the SpringBoot app (on the same level as src folder, pom.xml etc.) simply called Dockerfile (no extension) and put in the following:

FROM java:8
EXPOSE 8080
ADD target/MyBuiltApp.jar app.jar
ENTRYPOINT ["java","-jar","app.jar"]

Build Docker Image

Instruct Docker to build the image, and tag it as mydockerizedservice:

NOTE: You must actually build your application in Maven first, otherwise there might not be anything to find at target/…

$ mvn -U clean install
$ docker build -t mydockerizedservice .

Run Docker Container

To initially run a new container from this image:

$ docker run -p 8765:8080 -t mydockerizedservice

To stop and re-run this container:

$ docker ps

This gives us the following Container ID for our service: 51ed23ae9b5f

Now stop the service:

$ docker stop 51ed23ae9b5f

And re-start the existing Container:

$ docker start 51ed23ae9b5f

And that is pretty much it. I’m no Docker expert, evidently, but I’m happy to answer any questions related to this. And I’m spent.

 

TRP

Re-wiring The Grid

mess_of_wires

I’ve been doing some optimisation and “streamlining” of the Gee-Oh! Android library code, including doing away with an over-engineered “in memory” store of grid squares, and it has come with its own (not entirely unexpected) problems.

A good thing is that I’ve realised the kickstartFailedDownloads() method has been masking a multitude of sins.

Anyway, I now find myself in the process of re-wiring the core grid code quite heavily. I’m currently trying to figure out why – after animating to location – we are left with a partially populated grid where some grid squares never seem to get past the “initialised” state.

So this means a ton of System.out.println nonsense (who uses proper logging??) that I will remove once the investigation is complete. Got to get to the bottom of this!

After I’ve definitely fixed all the issues that have been previously masked, I will plug kickstartFailedDownloads() back in and try to tie up any further loose ends with the grid setting code, post-“in memory grid removal”. I’m sure it will all go smoothly…

 

TRP