Components of a micro service
A micro service is no different from a service in a service orientated architecture. It is a standalone application with an API that is defined. Each API call performs functionality such as updating the database, retrieving data from the database, creating files, sending messages etc.
In our application we will be supporting a REST API and a MessageQ API.
The REST API uses the HTTP protocol. This means our application must have a built in web server that can process HTTP messages and output JSON formatted data. All synchronous requests will be sent using the REST API. Our API must use HTTP status codes correctly, parse the header records correctly, implement security with bearer tokens (explained in a later blog post). Every service that uses our API must be able to process successful and unsuccessful response codes gracefully.
The MessageQ API uses a queue name and a message with an ID and a payload. All messages will be sent in JSON format. This means our application must have a built in message queue consumer which can process the next message in a queue concurrently with other messages being processed. All asynchronous requests will be sent using message queues.
The reason for using message queues for asynchronous data is to allow for rapid scaling. For example if we need to send 20,000 statements in a day we could have 10 instances of our documentServer service creating 2000 statements each which will run much quicker than one instance processing 20,000 statements.
What is a service
Thank heavens for Docker. This amazing tool allows us to wrap up our application into a single container where that container listens either to a message queue or on a HTTP port for an API call.
The Docker container will contain:
- a small web server
- a message queue interface for reading and creating messages
- application logic for processing API requests
At a minimum we could just run a Docker container with a single server for each of our components. Each docker container runs on your server linked to a different port. So if the clientApp was on port 8080 and the edgeServer was on 8300 and intefaceServer 8301 etc. we could run our full app using just Docker.
client------>clientApp------>edgeServer
externalVendors--->interfaceServer
accountService
documentService
dataStoreService
authenticationService
By pointing a web browser to the web browser client app at port 8080 we could run our entire application just using Docker.
Our services need to be able to discover where the other services they will communicate with exist. In this scenario they will exist on the host server ar ports 8080, 8300, 8301 etc. This information will need to be in a common config data store or in a minimal core library or in a service lookup container.
So in our simplest form, our micro services are each docker containers where he docker container includes a web server, a message queue interface and application logic to process API requests. Different containers may include other code, for example a emailService may be configured with email software and parameters, an ivrService might be configured with hooks into a phone system.
Scaling the App
We have choices to scale our app. The obvious choices are to use Docker Compose and Docker Swarm or to use Kubernetes. For enterprise apps, Kubernetes seems to be more popular though arguably it adds more levels of complexity.
Kubernetes is an orchestration tool. What this means is you tell it what you want to happen and it makes it happen. So if I want 10 instances of my documentService running to print the statements more efficiently I tell kubernetes to make sure 10 instances are running and it will do that. If an instance crashes or stops unexpectedly Kubernetes will just restart it. This is good and bad. It means you are not searching for failures constantly but you must build your app in such a way to gracefully handle restarts.
Kubernetes requires quite a bit of configuration. Not only are you configuring nodes and pods, and which instances of your containers will run in pods, you are configuring services which work like load balancers. We will send our requests to the documentService service as defined by Kubernetes and it will decide which pod running our documentService container to send it to, based on routing configuration you have configured (round robin etc.)
As a developer you have to understand Kubernetes sees your app as something perishable where nothing inside the container needs to stay around. This means where you previously may have written logs to a file, they now get directed to a log aggregator (like Splunk) and any files you create inside the container disappear constantly. A good rule of thumb is you dont create any files inside the container unless it is in a docker volume, where that volume is an NFS drive mapped onto each container. For the most part though we want to avoid files and try to deal with Blobs (binary large objects stored in a data store).
Datastore
Each micro service is a stand alone app which means it has its own datastore. The reality of this is somewhat dubious because you might have 15 to 20 micro services and do you really want to maintain 15 or 20 different datastores.
A compromise where the data store is an RDBMS is to use a dedicated schema for each micro service all hosted in a common database. Your DBA will thank you but you still have enough isolation that one micro service will not affect another. Unless of course you have database maintenance of the whole database affecting multiple schemas. A good redundancy scheme can mitigate this where a mirror database will kick in if the master if offline. So we will not consider it a problem in or design.
Some micro services may been to write Blobs (use MongoDB or AWS), some may need quick text searching (use ElasticSearch), some may not need data stores at all.
ORM or SQL
Its fair to say an ORM can save you time if you use it for change management in every release but I have never been a fan. I find you always need to drop down to SQL for more complex queries or large data loads for most efficiency and some of the ORM functionality (transaction write behind, cascading updates) eventually cause you to spend more time chasing exceptions, deadlocks and non-committed data than you would if you use standard SQL.
One disadvantage of SQL is it is not easy to change database vendors halfway through an implementation but I have never seen this happen anyways. Typically if you are an Oracle shop you are staying with Oracle and this is also true for SQL Server or MySQL. The latter is the most likely to be updated to Postgres or Oracle but these updates tend to be fairly simple.
The other big question in large projects is who owns the schema. The Project technical leads and the DBA must own the schema and the developers must go through a process for changes. A good database design will always evolve a little as the project scope changes (especially for large multi month or multi year projects). Business is always growing and changing and scope will do the same.
I find it quite sad that many junior developers are coming through without SQL experience. This is a foundation skill that every developer needs to have. I encourage you to learn it.
Comments
Post a Comment