Refresher: What is GraphQL

GraphQL is a query language for APIs and a runtime for fulfilling those queries with your existing data. GraphQL provides a complete and understandable description of the data in your API, gives clients the power to ask for exactly what they need and nothing more, makes it easier to evolve APIs over time, and enables powerful developer tools.

So GraphQL has two major components :

• Schema : GraphQL server uses a schema to describe the shape of request / graphs. This schema basically defines a hierarchy of types with fields that are populated from back-end data stores.
• Resolvers : Server needs to know how to populate data for every field in your schema so that it can respond to requests for that data. To accomplish this, it uses resolvers. Resolvers can be of three types-
  • Queries
  • Mutations
  • Subscriptions

To know more, please read this post “Building GraphQL API with Nodejs, TypeGraphQL , Typegoose and Troubleshooting common challenges”.

The backstory

A year ago, when I started working at Tekie. We were more focused into building and shipping our MVP (Minimum Viable Product), Both the User facing and Internal application. We had no issues with the current architecture i.e Nodejs and GraphQL. But as we kept growing and getting more and more users onboard we started to face some latency issues from our GraphQL API.

Adoption of new features were really fast and but as our mongoDB collection size started to grow and queries kept getting more complex, latency issues were a bigger problem now. We started to scale out by increasing our MongoDB clusters and autoscaling backend infrastructure based on kubernetes and AWS. And this is exactly what you don’t want to do and burn cash at blazingly fast pace.

The usual…

Now we quickly turn our heads to optimizations from development And the first thing we explored was Dataloader as every GraphQL developer would do to reduce database trips.

After a while we found out that the way in which our backend architecture was configured it was not possible to use dataloader. Every field which we requested where actually getting resolved before the result was returned i.e Using graphQL’s field.resolve we would resolve each and every field for every individual object that existed in an array. So this was always on the nextTick of nodejs and dataloader couldn’t batch and cache those requests.

The sqaure one

Okay, After a lot of experimentation and reflection we started to test out few Queries with MongDB Aggregation. And the results are simply amazing, we had a reduction of almost 80% in our latency test.

We quickly jumped into integrating aggregation pipelines into our generic code so that every query can be optimized. As part of the plan we first decided to build a helper library to easily build aggregation pipelines but there already existed a similar library called mongodb-pipeline-builder we quickly forked the project and started building our custom solutions on top of that already existing library and published a new version mongodb-aggregation-builder.

The showdown…

As soon as we had the helper library in place, we started integrating the logic with the help of that library into our generic backend architecture. This was a major change to our architecture which can lead to production bugs. So we decided to create a custom directive called databaseController on Schema Type which would accept the mode of query.

type Topic @model @databaseController(mode: "aggregation") {
        order: Int! 
    title: String! @trim
    description: String @trim
    status: ContentStatus! @defaultValue(value: "unpublished")
}

So now we were able to control which type would be allowed to query using aggregation pipeline. With this in place we were able to transform following query as seen below.

GraphQL Query:

query {
    topics(filter: { status: published }, order_by: createdAt_DESC) {
        id
        title
        courses(filter: { status: published }, first: 1, order_by: createdAt_DESC) {
            id
            title
        }
    }
}

Aggregation Pipelines :

[
    { $match : { status: "published" } },
    { $sort: { createdAt: -1 } },
    { $limit: 1000 },
    { $lookup: {
      from: "Course",
      let: { coursesId: "$courses.typeId" },
      pipeline: [
         { $match: { $expr: { $in: ["$id", "$$coursesId" ] } } },
         { $match: { status: "published" } },
         { $sort: { createdAt: -1 } },
         { $limit: 1 },
         { $projection: { id: 1, title: 1 } }
       ]
       as: "courses",
      } 
    }
    { $projection: { id: 1, title: 1, courses: 1 } }
]

As a result we were able to reduce the complex query latency by an average of 50% and the number of database calls was just fired once per query 🤯 🚀 compared to 60 - 70 database calls earlier.

The more the merrier…

With the Aggregation Implemented we placed our GraphQL endpoint behind the GraphCDN services. Now with the help of GraphCDN we were able to cache our static content at an edge location. Which resulted into fewer queries hitting our backend as only dynamic data would be required and static content was always served from cache.

What's next

We were able to scale or optimize our GraphQL Backend latency and different issues. But there’s always few trade off we have to make. After a while we knew that lookup stage in aggregation pipeline are very costly operation if over used, so at that moment we just try to limit fetching relational fields.

But this continues to be a challenge for us. We now look forward to integrate Redis on Query Level with Aggregation Pipelines, architectural restructure to support Dataloader or Complex Filters inside Dataloader with Aggregation.

The Possibilities Are Endless 🔥…

Thanks for reading, stay awesome! ❤

What are Micro-frontends Anyway

If you are a backend developer, you might be already familiar with micro-services architecture. But for the front-end developers, it's not that common yet.

Micro-frontends are basically an architectural pattern in which a front-end codebase is decomposed into individual "microapps" consisting of different frameworks / libraries working loosely together, Obviously inspired by backend micro-services.

We could have something like this :

If you notice each microapps can be built with different frameworks or libraries and each microapps can be owned by a single team.

Why Micro-frontends ?

When I was first introduced with micro-frontends honestly it felt a bit off for me and so is to @dan_abramov

But as I started getting familiar with the concept, I realized there are multiple reasons for using micro-frontends. One of the biggest reason would be when migrating from old codebase to a new framework or library.

Problems with traditional monolithic architecture

In early stages of an application it might not be a big problem, But when application grows, the number of teams increases then it becomes a real challenge . When project is massive it becomes difficult to work with multiple people/teams. Unnecessary codebase being available due to which build time is increased. When using same micro-service API it is difficult to make changes when API responses are refactored.

Micro-frontends to the rescue

Adopting micro-frontend approach we could solve some of those problems like assigning individual team with core modules. As unnecessary codebase is removed it is easy to deploy and build time also decreases. individual microapps can be developed, tested, and deployed independently, separate modules can be inserted , removed, or replaced without any refactors, different modules of the frontend can be created using different frameworks or libraries. As organization requirements changes we can easily replace old modules with new ones. So we will continuously evolve our codebases.

Some of the big companies are already using it, to name few Spotify, Klarna, Zalando, Upwork, AirBnB, and Facebook.

Different approaches towards micro-frontends

Well while adopting micro-frontend architecture we have few approaches which can be used :

• Using a framework

  We can use Single-SPA which lets you combine multiple Frameworks or libraries in a single page.

• Using Web Components

  Web components can be hosted on different servers with different frameworks or libraries. Chris Kitson has discussed it in depth here : Creating Micro-frontends using Web Components (with support for Angular and React)

• Using a separate micro-service for each micro-frontend modules, which can be very beneficial when API changes.

Challenges in micro-frontends

As it comes with many advantages it also have some disadvantages too, while application grows we would be left with tons of codebases to maintain and to maintain the same best practices among all the microapps. It would be very hard to run tests on the the entire codebase locally. While different microapps are hosted on different servers there might be some performance issue which costs user experience. Inter microapps communication can be very difficult and complex to work on.

Conclusion

So basically, micro frontends are all about splitting up big modules into smaller chunks i.e. more manageable modules, and then being explicit about the dependencies between them. According to me Micro-frontend are the future we are heading into but there's still a long way to go. It certainly depends upon the organization requirements and how big the development team is. I hope you have enjoyed this article and got a sense of what micro-frontends really are and different approaches to it.

Thanks for reading, stay awesome! ❤

Feel free to follow me on LinkedIn. Do check my website @ smithgajjar.dev

So It’s probably worth posting out this article so that it could be an help for other developers and to briefly explain some of the codes and concepts around GraphQL.

I have created an boilerplate GraphQL API with Node.js, Apollo, TypeGraphQL , TypeScript , Nx, MongoDB , typegoose. You may want to check that out, here's the repository github.com/DevUnderflow/nx-node-apollo-grahql-mongo

What is GraphQL?

GraphQL is a query language for APIs and a runtime for fulfilling those queries with your existing data. GraphQL provides a complete and understandable description of the data in your API, gives clients the power to ask for exactly what they need and nothing more, makes it easier to evolve APIs over time, and enables powerful developer tools.

Basically it is new API standard that enables a better way of implementing API's and a great alternative to REST. In REST we need maintain a bunch API endpoints, While in GraphQL there is just a single API endpoint created.

So GraphQL has two major components :

• Schema : GraphQL server uses a schema to describe the shape of request / graphs. This schema basically defines a hierarchy of types with fields that are populated from back-end data stores.

  type User {
     username: String
     email: String 
  }
  

• Resolvers : Server needs to know how to populate data for every field in your schema so that it can respond to requests for that data. To accomplish this, it uses resolvers. Resolvers can be of three types-

  • Queries

    {
      user {
          username
          email
      }
    }
    

  • Mutations

    mutation {
     createUser(username: "test", email: "test@test.com") {
         username
         email
     } 
    }
    

  • Subscriptions

    subscription {
      users
    }
    

Why choose GraphQL over REST

Challenges with REST

We have been using REST API's for a long time now. But there exists some problems with REST like fetching data from REST API comes with lot of unnecessary data being fetched. Also there is a need to remember a lot of API endpoints. With GraphQL we describe in the client which data we want to have instead of just asking all the data.

GraphQL Solves the issue

In the backend, we need to define our data types which will form our schemas. and the resolvers to resolve the request coming frontend. In GraphQL we only need to maintain a single endpoint where we would request only those data that we need on the frontend.

GraphQL basically reduces the network call by enabling us to fetch all the data that we need in a single query from a single endpoint. GraphQL also comes with a set of challenges which we would discuss later in this article.

Using Apollo with TypeGraphQL

While the most used library for GraphQL is Apollo. Apollo Server comes with a lot of features like caching , external data loader and many more. The Apollo Client is a tool which helps you use GraphQL in the frontend. It provides different features like in-memory caching, state management etc. It can be integrated with multiple JavaScript frameworks like React, React Native, Vue, Angular and for iOS and Android, there are also possibilities to use the Apollo Client.

TypeGraphQL

TypeGraphQL is modern framework for creating GraphQL API's. While apollo is really great and solves many problems that we have. But developing a GraphQL API in Node.js with TypeScript is sometimes a bit of a pain. This makes it easy to use with typescript as your first language.

Creating a sample resolver would look like :

@Resolver()
class UserResolver {
  private userCollection: User[] = [];

  async users() {
    return await this.userCollection;
  }
}

Using TypeGraphql with Typegoose

Well Typegoose is basically a TypeScript wrapper around mongoose. It is easy to use in typescript environment and provides a lot features.

The first challenge I encountered while Integrating typegoose with typegraphql is that I had to define multiple interfaces one for typegoose, then one for typegraphql schema, So there was lot of redundancy happing around. So the solution I found was just to use typgraphql decorators on top of typegoose methods.

@ObjectType()
export class User {
  @Field()
  readonly _id: string;

  @Field()
  @prop()
  public username: string;

  @Field()
  @IsEmail()
  @prop()
  public email: string;
}

So here we define our mongoDB models with prop() decorator from typegoose and along with we simultaneously define our GraphQL schemas with ObjectType() and Field() decorators form typegraphql. It basically removes all the redundant interfaces we had earlier.

The second challenge I encountered is that during the initial phase I was writing all my core logic directly into resolver methods which eventually created lot of problems in maintaining the codebase So the solution was that I started refactoring all my codebase into different folder structure. I started using typescript decorator features for services and used dependency Injector to inject all my Database models and Services directly into Resolvers.

import { Service, Inject } from "typedi";
@Service()
export class UserService {
  constructor(@Inject("userModel") private readonly UserModel) {}
  async getAll() {
    return this.UserModel.find();
  }
}

So here we are creating UserService which injects userModel. now once we have our service running we can inject this directly into our Resolvers as :

@Resolver()
export class UserResolver {
  constructor(
    private readonly userService: UserService,
  ) { }
  @Query(() => [User])
  async users() {
    return this.userService.getAll();
  }
}

Most Common Challenge you'll face

When using GraphQL we have a lot advantages but it comes with its own set of challenges or cons you would say. The most common problem you would face is that server making a lot of multiple request to the database then expected. Suppose you have a list of posts, in which each of post has a user document to it. Now you may want to fetch all of these posts, with user data. When using REST this would be like having two database calls. One for posts and one for users corresponding to these posts.

But what in GraphQL? we now have an extra call to fetch each user data per resolver that is per. Now let's say we have 10 posts and each post also has 5 comments, each of which has an user document. So the number of calls we'll have is one for the list of posts, 10 for the post authors, 10 for each sub-list of 5 comments and 50 for the comment users which sums up to around 71 database calls to fetch a set of data!

Nobody would want this to happen, waiting 15 secs to load a set of posts. To solve this problem we have Dataloader library.

Dataloader basically lets will let you combine or batch multiple similar requests and cache database calls. Now the dataloader detects that posts having similar id's and batch them together and will reuse the user document which it already has in memory instead of making a new database call.

So These were the challenges I have came across till now while building a GraphQL based API.

The source code for the Nx-Node-Apollo Graphql API is here github.com/DevUnderflow/nx-node-apollo-grahql-mongo

Thanks for reading, stay awesome! ❤

I hope you have enjoyed this article and you may avoid above problems beforehand , I also hope that if it gave you some sort of inspiration for your work.

If you may want to checkout other articles its right here blogs.smithgajjar.dev. Feel free to follow me on LinkedIn.

Do checkout my website at smithgajjar.dev.

Let's focus on why this is my default go-to language and why I prefer to write code using TypeScript rather than JavaScript.

TypeScript - JavaScript With Superpowers

Believe it or not but typescript has saved me a lot of time by catching errors and providing fixes before actually running the code.

Let's see how

Did you see that little red underline? That’s what TypeScript giving us a hint that something has gone wrong while writing code. You may have already figured out what's wrong here. Number has been given to user method instead of a string.

That’s the reason why I love TypeScript ❤.

Moving on, there's lot more to typescript than just type checking. Most important features I use every day is Decorators and interfaces

Decorators

Decorators are special kind of declaration or method that can be attached to a class, method, property, or any parameter. Decorators use the @decorator_name, where decorator_name must evaluate to a method that will be called at runtime with information about the decorated declaration.

Interfaces

Providing a type shape to JavaScript objects.Interfaces are the powerful way of defining contracts within your code as well as contracts with code outside of your project.

Typescript works great with VS Code. Anyways official Typescript docs is a great way to learn language in detail.

How To Get Started With Typescript

It is available as a package on the npm registry available as "typescript".

Node.js environment is must to run the package. Then dependency manager like npm, yarn can be used to download TypeScript into your project.

npm install -g typescript

You can then run the TypeScript compiler using one of the following commands:

npx tsc

Now we initialize a new typescript project using

tsc --init

You should end up with the TypeScript config.

Now all you need is to create a .ts file and start writing code in typescript. Once you are done you can compile .ts file into .js using following command

tsc filename.ts

Great 🎉 Now you can start building something awesome with Typescript.

If you have already been using it and wants to start using typescript in your NodeJS application I have a Nodejs / Typescript starter project built using Nx workspace and express server includes MongoDB and jest testing framework. If you like the project leave a star🌟

Github Repository : nx-node-typescript-api

To conclude

That's why I love TypeScript. Basically, Typescript allows you to write better, cleaner, productive and more maintainable code. Obviously, you don’t have to go all out for TypeScript directly and start using decorators, interfaces or classes. Initially I have used TypeScript only with type checking on small projects. and gradually you can start adding more and more features.

I hope this post helps you decide if you should give it a try, and I strongly encourage you to start using it.

Thanks for reading, stay awesome! ❤

Do check my website smithgajjar.dev Follow me on LinkedIn

Deno v1.0 has launched and I think that it may be on the right track to replace Node.js in the future.

Deno

Deno is a secure runtime for JavaScript and TypeScript. Imagine if you could write TypeScript without any config files, and bundle it all together into a single ES Module where both the TypeScript support and bundler are present in the core. That’s what it feels like when you get started with Deno.

It’s a modern and secure runtime for JavaScript and TypeScript that uses V8 and is built in Rust. Whereas Node.js is written in C++ and JavaScript.

Fun fact: Deno is an anagram of Node. If you sort() node it becomes deno🦕.

Deno ships with many features required for writing modern JavaScript & TypeScript, and WebAssembly code.

• 📦 bundler
• 🐛 debugger
• 🤖 test runner
• 🧶 code formatter
• 📖 docs generator
• 🧵 WebAssembly support

Deno has some interesting features

• Secure by default.
  No file, network, or environment access, unless explicitly enabled.
• Single Executable.
  Ships only a single executable file.
• TypeScript Support Deno ships with out of the box TypeScript compiler.
• Module system No package.json, no node_modules. Source files can be imported using a relative path, an absolute path or a fully qualified URL of a source file:

  import { test } from "https://unpkg.com/deno_testing@0.0.5/testing.ts"  
  import { log } from "./util.ts"
  

What are the main issues with Node.js?

• Any program can write to the filesystem and the network This might be a security problem, especially when intalling untrusted packages from npm. The [crossenv](https://blog.npmjs.org/post/163723642530/crossenv-malware-on-the-npm-registry) incident is an example. If crossenvhad not had writing permissions, this would not have happened.

• The build system (GYP) Using GYP to build a module that links to a C library is a big pain. In order to have a sane DX you’ll have to use node-gyp (a layer on top of GYP) and maybe other layers (like [nan](https://www.npmjs.com/package/nan)).

• The module system and npm The main problem here is that the module system isn’t compatible with browsers so our code isn’t fully isomorphic. This is mainly caused by two reasons: storing dependencies in node_modules and having a package.json.

Let's get started with installing Deno

Using PowerShell (Windows):

iwr https://deno.land/x/install/install.ps1 -useb | iex

With Shell:

curl -fsSL https://deno.land/x/install/install.sh | sh

With Homebrew:

brew install deno

Now check if deno was installed by running the deno --version command in your terminal.

Simple http server

This example contains a simple http server (app.ts):

 import  { serve }  from  "https://deno.land/std@0.50.0/http/server.ts";
 const s =  serve({ port:  8000  });
 console.log("http://localhost:8000/");
 for  await  (const req of s)  {
    req.respond({ body:  "Hello World\n"  });
 }

Run the code:

 deno run app.ts

This results into permission error

 error: Uncaught PermissionDenied: read access to "http/server.ts", run  again with the --allow-read flag

 ► $deno$/dispatch_json.ts:40:11
 at DenoError ($deno$/errors.ts:20:5)
 ...

This is because deno allows you to control the granularity of permissions. To run above application you need set some flags indicating deno that particular permission are allowed.

 deno run --allow-net app.ts
 > http://localhost:8000/

Now open up your browser at localhost:8000. You will be see the Hello World text. Okay this was just a basic demonstration of how you could create simple http server using deno.

See more example here

I have created User REST API in deno feel free to check it out here. Clone the Repo and play around. Contribution are always welcome🎉

Conclusion

There's still a long time until Deno reaches a production-ready state, but I think it’s on the right path in order to be a better Javascript runtime than Node.js🔥. Thanks for reading! 👋👋

Do check my website smithgajjar.dev and GitHub here Follow me on LinkedIn