Type-Safe Lightweight DDD with
Effect Schema

Yuichi Goto | JSConf JP on November 23, 2024

Who I am

  • Yuichi Goto
  • @_yasaichi
  • @yasaichi.bsky.social
  • @yasaichi

Professional Background

  • Backend Engineer since 2015
  • Co-author of Perfect Ruby on Rails
  • Working at EARTHBRAIN since 2023

About EARTHBRAIN

  • Founded in July 2021
  • 200 employees (including 50 developers)
  • Provides Smart Construction® solutions

About Smart Construction®

  • A digital solution enhancing safety, productivity, and sustainability in the construction industry
  • Integrates interconnected software and hardware products
  • Used in over 20 countries worldwide

Background and Purpose

  • This presentation is based on my recent experience rewriting a legacy API gateway, which is now in production.
  • The new API gateway leverages Deno, NestJS, and Effect.
  • In this presentation, I will:
    • Explore potential challenges in adopting Effect
    • Demonstrate our practical approach to Effect

Agenda

  1. Effect: A Concise Overview 👈
  2. Challenges in Effect Adoption
  3. Our Practical Approach
  4. Conclusion

What is Effect(-TS)?

  • A cutting-edge TypeScript library that became stable in April this year [1]
  • Enables building typed, robust, and scalable applications using the Effect System [2]—a concept in computer science [3].
  • Provides standard library-like features (which we'll explore in detail later)

The Effect type: powerful Promise

Effect<Success, Error, Requirements>

  • Success:: Represents the success type
  • Error: Represents the error type
  • Requirements: Represents required dependencies

Creating Effects

import { Cause, Effect } from 'effect';

Effect.succeed({ id: 1n, name: 'yasaichi' }) satisfies Effect.Effect<User>;

const getUserRequest = (
  userId: bigint
): Effect.Effect<any, Cause.UnknownException> =>
  Effect.tryPromise(() =>
    fetch(`${API_ORIGIN}/users/${userId}`).then((res) => res.json())
  );

const parseUser = (data: unknown): Effect.Effect<User, Error> =>
  Effect.try({
    try: () => User.parse(data),
    catch: () => new Error('Failed to parse user'),
  });
Default error type

🤔💭 Require manual wrapping?

Running Effects

// Unlike Promises, you must explicitly run effects using these methods.
await Effect.runPromise(getUserRequest(1n));
// => { id: "1", name: "yasaichi", ... }

// You do not use `runSync` usually because there is no way to know in advance
// if an effect will execute synchronously or asynchronously.
Effect.runSync(parseUser({ id: '1', name: '' }));
// throws (FiberFailure) Error: Failed to parse user

Composing Effects

// Using Pipe and Functor/Monad
const getUserByIdUsingPipe = (
  userId: bigint
): Effect.Effect<User, Error | Cause.UnknownException> =>
  getUserRequest(userId).pipe(Effect.andThen(parseUser));

// Using Generator and yield*
const getUserByIdUsingGenerator = (
  userId: bigint
): Effect.Effect<User, Error | Cause.UnknownException> =>
  Effect.gen(function* () {
    const res = yield* getUserRequest(userId);
    return yield* parseUser(res);
  });
The errors are combined into a union type.

🤔💭 Functor, Monad, yield*?

Agenda

  1. Effect: A Concise Overview
  2. Challenges in Effect Adoption 👈
  3. Our Practical Approach
  4. Conclusion

Two potential challenges

  • The need for an additional adapter layer
  • A shift to a different programming paradigm

Additional adapter layer

Issue: Manual wrapping of existing functions with Effect

const getUserRequest = (
  userId: bigint
): Effect.Effect<any, Cause.UnknownException> =>
  Effect.tryPromise(() =>
    fetch(`${API_ORIGIN}/users/${userId}`).then((res) => res.json())
  );

await Effect.runPromise(getUserRequest(1n));
// => { id: "1", name: "yasaichi", ... }
This is because the Effect value is not a built-in JavaScript object.

Examples of functions requiring wrapping

  • Database I/O: db.insert (Drizzle) , sql (postgres)
  • File I/O: fs.readFile, fs.writeFile
  • Network I/O: fetch, http.get
  • Storage I/O: localStorage.setItem
  • Any other functions that indirectly perform these operations

Solution: Introducing interface adapters

import { FetchHttpClient, HttpClient, HttpClientError } from '@effect/platform';
import { Effect } from 'effect';

// `HttpClient` is an adapter of the `fetch` function
const getUserRequest = (
  userId: bigint
): Effect.Effect<unknown, HttpClientError.HttpClientError> =>
  HttpClient.get(`${API_ORIGIN}/users/${userId}`).pipe(
    Effect.andThen((res) => res.json),
    Effect.scoped,
    Effect.provide(FetchHttpClient.layer)
  );

await Effect.runPromise(getUserRequest(1n));
// => { id: "1", name: "yasaichi", ... }

Examples of adapter implementations

  • @effect/platform: Provides adapters like Command, FileSystem, HttpClient, KeyValueStore, and more
  • @effect/sql: Offers SQL client adapters for mysql2, postgres, drizzle-orm, and more

Effect adoption ≈ New runtime adoption

🤔💭 While the adapters address the initial issue, they raise new concerns ...

Different programming paradigm

Issue: Potentially unfamiliar concepts for your teammates

// Using Pipe and Functor/Monad
const getUserByIdUsingPipe = (
  userId: bigint
): Effect.Effect<User, Error | Cause.UnknownException> =>
  getUserRequest(userId).pipe(Effect.andThen(parseUser));

// Using Generator and yield*
const getUserByIdUsingGenerator = (
  userId: bigint
): Effect.Effect<User, Error | Cause.UnknownException> =>
  Effect.gen(function* () {
    const res = yield* getUserRequest(userId);
    return yield* parseUser(res);
  });

Two primary ways of composing Effects

  • Using pipe: Combines Effect.andThen and Effect.all to manage control flow and concurrency
  • Using Effect.gen: Works with the yield* keyword to delegate generator* iteration to an underlying iterator
* Effect doesn't use generators internally [4]; it implements the iterator protocol.

The underlying concepts

  • pipe: F# pipes (cf. Hack pipes, currently at TC39 Stage 2 [5])
  • Effect.gen: Iterators and Generators, commonly used but often hidden in everyday programming
  • Effect.andThen: Functors and Monads*, concepts that many developers find challenging
*[IMO] The Effect documentation appears to intentionally avoid using these terms.

What happened in our recent project

  • Partial adoption of the Effect type and value in specific areas (e.g., NestJS Guards)
  • Team faced challenges with Effect code readability and composition.
  • Using pipe with Functors and Monads proved more complex than expected.

Agenda

  1. Effect: A Concise Overview
  2. Challenges in Effect adoption
  3. Our Practical Approach 👈
  4. Conclusion

Recap so far

  • The Effect type's powerful expression and composability enable building type-safe applications.
  • The Effect value is not a built-in JavaScript object, which leads to two potential challenges:
    • The need for an additional adapter layer
    • A shift to a different programming paradigm

The other Effect features still shine

  • Option and Either (Result)
  • Pattern matching
  • Schema validation (previously provided by @effect/schema)
These features work together smoothly.

Creating Option and Either values

import { Either, Option } from 'effect';

const some: Option.Option<number> = Option.some(42);
const none: Option.Option<number> = Option.none();

const right: Either.Either<number, Error> = Either.right(42);
const left: Either.Either<number, Error> = Either.left(
  new Error('Not a number')
);

Narrowing Option and Either types

const getOrZero = (option: Option.Option<number>): number => {
  if (Option.isNone(option)) {
    option satisfies Option.None<number>;
    return 0;
  }

  option satisfies Option.Some<number>;
  // => { _id: "Option", _tag: "Some", value: ... }

  return option.value;
};

getOrZero(Option.some(42)); // => 42
The “_tag” field is conventionally used for discriminated unions.

Pattern matching

import { Match, Option } from 'effect';

const getOrZero = (option: Option.Option<number>): number =>
  Match.value(option).pipe(
    // Match when the '_tag' property is 'None'
    Match.tag('None', () => 0),
    // Match an object with a numeric property 'value'
    Match.when({ value: Match.number }, ({ value }) => value),
    // Ensure all possible patterns have been accounted for
    Match.exhaustive
  );

getOrZero(Option.none()); // => 0

Defining schemas (similar to Valibot)

import { Schema as S } from 'effect';

const User = S.Struct({
  id: S.BigInt.pipe(S.positiveBigInt()),
  // Shorthand for `S.String.pipe(S.nonEmptyString())`
  name: S.NonEmptyString,
})

interface User extends S.Schema.Type<typeof User> {}
// { readonly id: bigint; readonly name: string; }
Readonly types by default

Decoding and Encoding concept in Effect

Decoding and Encoding examples

// Decoding a user from query parameters etc.
S.decodeUnknownSync(User)({ id: '1', name: 'yasaichi' });
// => { id: 1n, name: "yasaichi" }
S.decodeUnknownEither(User)({ id: '1', name: '' });
// => { _id: "Either", _tag: "Left", left: { _id: "ParseError", ... } }

// Constructing a user from a DTO etc.
// If the object does not match the schema, an error will be thrown.
const user: User = User.make({ id: 1n, name: 'yasaichi' });
// => { id: 1n, name: "yasaichi" }

// Encoding a user when persisting data in database etc.
S.encodeSync(User)(user);
// => { id: "1", name: "yasaichi" }

Our recent approach to Effect

  • Leveraging features unrelated to the Effect type, thereby avoiding the challenges discussed earlier
  • Implementing lightweight Domain-Driven Design (DDD) with this approach to maximize type safety

Example: Resending site invitations in our product

State transition of site invitations

⚠️ Some details (e.g., AuthZ) are intentionally omitted for clarity.

Entity and Value Object

Defining Entity fields with Branded Types

import { validate } from 'email-validator';

// In domain/values/email.value.ts
export const Email = S.String.pipe(S.filter(validate), S.brand('Email'));
export type Email = S.Schema.Type<typeof Email>;

// In domain/entities/site-member.entity.ts
export const SiteMemberAccessLevel = S.Literal(
  'Manager',
  'Worker',
  'Viewer'
).pipe(S.brand('SiteMemberAccessLevel'));
export type SiteMemberAccessLevel = S.Schema.Type<typeof SiteMemberAccessLevel>;
These values are distinguished from string at the type level.

Defining the base fields with Effect Schema

import { Email } from '../values/email.value.ts';
import { SiteMemberAccessLevel } from './site-member.entity.ts';
import { SiteId } from './site.entity.ts';

export const SiteInvitationId = S.PositiveBigInt.pipe(
  S.brand('SiteInvitationId')
);

const siteInvitationFields = S.Struct({
  id: SiteInvitationId,
  siteId: SiteId,
  email: Email,
  accessLevel: SiteMemberAccessLevel,
  roles: S.optionalWith(S.NonEmptyString, { as: 'Option' }),
}).fields;

Utilizing Schema Class APIs for custom methods

export class PendingSiteInvitation extends S.TaggedClass<PendingSiteInvitation>()(
  'PendingSiteInvitation',
  { ...siteInvitationFields, isResent: S.Boolean }
) {
  resend(): PendingSiteInvitation {
    return PendingSiteInvitation.make({ ...this, isResent: true });
  }
}

export class ExpiredSiteInvitation extends S.TaggedClass<PendingSiteInvitation>()(
  'ExpiredSiteInvitation',
  { ...siteInvitationFields, isResent: S.Boolean }
) {
  resend(): PendingSiteInvitation { /* The same as above */ }
}

Creating entity types using discriminated unions

export class AcceptedSiteInvitation extends S.TaggedClass<AcceptedSiteInvitation>()(
  'AcceptedSiteInvitation',
  siteInvitationFields
) {}

export type SiteInvitation =
  | PendingSiteInvitation
  | AcceptedSiteInvitation
  | ExpiredSiteInvitation;
Domain rule 1: Invitation status consists of three states.
Domain rule 2: Accepted invitations cannot be resent.

Advanced topic: Implementing equivalence

import { Equivalence } from 'effect';

const pendingOne: PendingSiteInvitation = PendingSiteInvitation.make({ ... });
const resentOne = pendingOne.resend();

S.equivalence(PendingSiteInvitation)(pendingOne, resentOne); //=> false

const SiteInvitationEquivalence = Equivalence.mapInput(
  Equivalence.bigint,
  (siteInvitation: SiteInvitation) => siteInvitation.id
);

SiteInvitationEquivalence(pendingOne, resentOne); //=> true
This is appropriate for Value Objects.

Repository

Defining interfaces

import { Option } from 'effect';
import type {
  SiteInvitation,
  SiteInvitationId,
} from '../../entities/site-invitation.entity.ts';

export interface SiteInvitationRepository {
  findUnique(
    siteInvitationId: SiteInvitationId
  ): Promise<Option.Option<SiteInvitation>>;
  save(siteInvitation: SiteInvitation): Promise<void>;
}

Implementing the interfaces

import { SiteInvitationRepository } from '../../domain/interfaces/...';

export class InternalPlatformSiteInvitationRepository
  implements SiteInvitationRepository
{
  constructor(
    private readonly internalPlatformApiService: InternalPlatformApiService
  ) {}

  async findUnique(...) { /* Implementation details shown in the next slides. */ }

  async save(...) { /* Implementation details shown in the next slides. */ }
}

Implementation details: findUnique method

async findUnique(siteInvitationId: SiteInvitationId) {
  try {
    const res = await this.internalPlatformApiService.getSiteInvitationById({
      siteInvitationId: S.encodeSync(SiteInvitationId)(siteInvitationId),
    });

    return Option.some(this.decodeSiteInvitationResponseSync(res));
  } catch (error) {
    if (error instanceof ApiException && error.code === 404) {
      return Option.none();
    }

    throw error;
  }
}
Track only recoverable errors via types.

Implementation details: save method

async save(siteInvitation: SiteInvitation) {
  await Match.value(siteInvitation).pipe(
    Match.tag('AcceptedSiteInvitation', () => { /* Out of scope */ }),
    Match.when({ isResent: true }, (resentOne) =>
      this.internalPlatformApiService.resendSiteInvitation({
        siteInvitationId: S.encodeSync(SiteInvitationId)(resentOne.id),
      })
    ),
    Match.orElse(() => { /* Out of scope */ })
  );
}

Application Service

Define DTOs using Effect Schema

export const CreateSiteReinvitationInputDto = S.Struct({
  siteInvitationId: S.PositiveBigInt,
});
export interface CreateSiteReinvitationInputDto extends S.Schema.Type<...> {};

export const CreateSiteReinvitationOutputDto = S.Struct({
  id: S.PositiveBigInt,
  siteId: S.PositiveBigInt,
  email: S.String,
  accessLevel: AccessLevelFromString,
  roles: S.optionalWith(S.String, { as: 'Option' }),
  _tag: S.propertySignature(TagFromStatus).pipe(S.fromKey('status')),
});
export interface CreateSiteReinvitationOutputDto extends S.Schema.Encoded<...> {}

Creating Service classes with DTOs

import { Either, Schema as S } from 'effect';
import { type SiteInvitationRepository } from '../domain/interfaces/...';
import { CreateSiteReinvitationInputDto } from './dto/...';
import { CreateSiteReinvitationOutputDto } from './dto/...';

export class SiteReinvitationService {
  constructor(
    private readonly siteInvitationRepository: SiteInvitationRepository
  ) {}

  async create(
    createSiteReinvitationInputDto: CreateSiteReinvitationInputDto
  ): Promise<Either.Either<CreateSiteReinvitationOutputDto, Error>> {
    // Implementation details shown in the next slide.
  }
}

Compositing Entities and Repositories

const siteInvitation = await this.siteInvitationRepository.findUnique(
  SiteInvitationId.make(createSiteReinvitationInputDto.siteInvitationId)
);

if (
  Option.isNone(siteInvitation) ||
  isTagged('AcceptedSiteInvitation')(siteInvitation.value)
) {
  return Either.left(new Error("Couldn't find SiteInvitation"));
}

const resentOne = siteInvitation.value.resend();
this.siteInvitationRepository.save(resentOne);

return Either.right(S.encodeSync(CreateSiteReinvitationOutputDto)(resentOne));

Summary

  • Our achievement: Implementing domain rules and recoverable errors at the type level, verifiable during static analysis
  • How we accomplished this:
    • Entity layer: Through Effect Schema and discriminated unions
    • Other layers: Using pattern matching and Option/Either types

Agenda

  1. Effect: A Concise Overview
  2. Challenges in Effect adoption
  3. Our Practical Approach
  4. Conclusion 👈

Let's start with the Effect schema,
not the Effect type.

Thank you

This presentation is created by Marp. Great thanks @yhatt!

References

  1. Effect 3.0 – Effect Blog,URL: https://effect.website/blog/effect-3.0↩
  2. effect/README.md at f7e4f21e4e6c866545a7bc2cb03cd8e43407c6a9 · Effect-TS/effect,URL: https://github.com/Effect-TS/effect/blob/f7e4f21e4e6c866545a7bc2cb03cd8e43407c6a9/README.md#effect↩
  3. Nielson, F., & Nielson, H.R. (1999). Type and Effect Systems. Correct System Design.↩
  4. Michael Arnaldi on X,URL: https://x.com/MichaelArnaldi/status/1784165004716945794↩
  5. tc39/proposal-pipeline-operator: A proposal for adding a useful pipe operator to JavaScript.,URL: https://github.com/tc39/proposal-pipeline-operator↩

Effect Schemaによる型安全な軽量DDDの実践

https://excalidraw.com/#json=qil3vXGPlmguecb7M6LLr,UQqg9IGYkMVWjQ4f-eh_Pw

https://excalidraw.com/#json=C6JJ9MemngR6ZO_DSbAKT,cJsRzp1D3zkYiLS5qchqPA

https://excalidraw.com/#json=nPaiW1QZx_q0oMQ5MuNPC,xf5oBAm6YPXJbHk4iq23fw