Result of the investigation.
Current realisation inserts new records one by one. Batching doesn't work because of IDENTITY id generator.

Hibernate batch processing works only with SEQUENCE id generator. And with properties:
spring.jpa.properties.hibernate.jdbc.batch_size=30
spring.jpa.properties.hibernate.order_inserts=true

Links: https://docs.jboss.org/hibernate/orm/4.3/manual/en-US/html/ch15.html

Solution:
Move to SEQUENCE id generator.
Create the my_table_id_seq for table my_table. Or create new in case there is not exist.
Then restart it from the latest existing primary key.

SELECT SETVAL('my_table_id_seq', (SELECT MAX(id) + 1 FROM my_table));

Test 1: saving of 50_000 records took 20 seconds.
Set up with identity generator:
1) MyEntity
  @Id
  @GeneratedValue(strategy = GenerationType.IDENTITY)
  @Column(name = "id")
  private Long id;

2) properties.yaml
spring.jpa.properties.hibernate.jdbc.batch_size=30
spring.jpa.properties.hibernate.order_inserts=true

In the log you can see that batchSize = 0. There is 50_000 logs entries. Batching doesn't work.

Log:

{"name":"Batch-Insert-Logger", "connection":5, "time":0, "success":true, "type":"Prepared", "batch":false, "querySize":1, "batchSize":0, "query":["insert into my_table (created,id,full_name,updated) values (?,?,?,?,)"], "params":[["2025-03-06 12:07:07.010665","698606","John Doe","2025-03-06 12:07:07.010666"]]}

Test 2: saving of 50_000 records took 6 seconds.
Set up with sequence generator:
1) MyEntity
  @Id
  @GeneratedValue(strategy = GenerationType.SEQUENCE, generator = "mySeqGen")
  @SequenceGenerator(name = "mySeqGen", sequenceName = "my_table_id_seq", allocationSize = 1)
  @Column(name = "id")
  private Long id;

2) properties.yaml
spring.jpa.properties.hibernate.jdbc.batch_size=30
spring.jpa.properties.hibernate.order_inserts=true

In the log you can see that batchSize = 30. There is 117 logs entries with batch size = 30 and 1 log entry with batch size = 20. Batching works.

Log:

{"name":"Batch-Insert-Logger", "connection":5, "time":0, "success":true, "type":"Prepared", "batch":true, "querySize":1, "batchSize":30, "query":["insert into my_table (created,id,full_name,updated) values (?,?,?,?)"], "params": []}

We'll say that a positive int divides itself if every digit in the number divides into the number evenly. So for example 128 divides itself since 1, 2, and 8 all divide into 128 evenly. We'll say that 0 does not divide into anything evenly, so no number with a 0 digit divides itself. Note: use % to get the rightmost digit, and / to discard the rightmost digit.

Example:
dividesSelf(128) → true
dividesSelf(12) → true
dividesSelf(120) → false

A pangram is a string that contains every letter of the alphabet. Given a sentence determine whether it is a pangram in the English alphabet. Ignore case. Return either pangram or not pangram as appropriate.

Example: The quick brown fox jumps over the lazy dog

The string contains all letters in the English alphabet, so return pangram.

Below is an example of a concurrent bank transaction system in Java. In this example, we model a bank account with methods for deposit, withdrawal, and a transfer operation that involves two accounts. We use a ReentrantLock in each account to ensure that modifications to the account balance are thread-safe, and we take care to avoid deadlocks when transferring funds between two accounts.

BankAccount Class:

• Fields & Lock:
  Each account has a unique id, a balance, and a dedicated ReentrantLock to protect its state.
• Deposit & Withdraw:
  The methods deposit() and withdraw() lock the account before modifying the balance to ensure thread safety.
• Transfer Method:
  The static transfer() method transfers funds between two accounts.
• To prevent deadlocks when acquiring two locks (one for each account), we always lock the accounts in a fixed order based on their IDs.
• Once both locks are acquired, the method checks for sufficient funds, performs the transfer, and then unlocks both accounts.

(Main Simulation):

• ExecutorService:
  We use a fixed thread pool to simulate concurrent transactions.
• Tasks:
  Three tasks are created to transfer funds between the accounts:
• Task 1 transfers money from Account 1 to Account 2.
• Task 2 transfers money from Account 2 to Account 1.
• Shutdown & Final Output:
  The executor is shut down gracefully after submitting the tasks, and once all tasks complete, the final balances of each account are printed.

This example demonstrates key concurrency concepts in a real-world banking scenario, including proper synchronization with locks, avoiding deadlock by enforcing a lock order, and managing concurrent tasks using an ExecutorService.

There are a lot of guides about setting up Prometheus module in NestJS application and several guides about usage kafka-prometheus-exporter library. The last one is not wideused for unknown reason for me. But there is no instructrusctions how to set up both.

Firstly, let’s intall and setup Prometheus module

npm install @willsoto/nestjs-prometheus prom-client

import { Module } from "@nestjs/common";

import { PrometheusModule } from "@willsoto/nestjs-prometheus";

@Module({

imports: [PrometheusModule.register()],

})

export class AppModule {}

At this stage you can run your application and visit localhost:3000/metrics to see default metrics.

The below snippet shows kafka client configuration and provides util methods for monitoring.

You don’t need to create new Registry() as it mentions in other guides. Just retrieve the existing ones and pass it to kafkajs-prometheus-exporter functions.

// kafka.config.ts

import {Consumer, Kafka, Producer} from 'kafkajs';

import {Injectable} from '@nestjs/common';

import {ConfigService} from '@nestjs/config';

import * as fs from 'fs';

import {

monitorKafkaJSAdmin,

monitorKafkaJSConsumer,

monitorKafkaJSProducer

} from '@christiangalsterer/kafkajs-prometheus-exporter';

import {Registry} from 'prom-client';

import * as promClient from 'prom-client';

@Injectable()

export class KafkaConfig {

private readonly _kafkaConfig: Kafka;

private readonly registry: Registry;

constructor(private readonly configService: ConfigService) {

this.registry = promClient.register

this._kafkaConfig = new Kafka({

brokers: configService.get('KAFKA_BROKER'),

clientId: configService.get('KAFKA_CLIENT_ID'),

});

let admin = this._kafkaConfig.admin();

monitorKafkaJSAdmin(admin, this.registry);

admin.connect().then(() => {console.log('Kafka admin connected');});

}

getConfig(): Kafka {

return this._kafkaConfig;

}

monitorConsumer(consumer: Consumer): void {

monitorKafkaJSConsumer(consumer, this.registry, { defaultLabels: {client_id: this.configService.get('KAFKA_CLIENT_ID')} });

}

monitorProducer(producer: Producer): void {

monitorKafkaJSProducer(producer, this.registry, { defaultLabels: {client_id: this.configService.get('KAFKA_CLIENT_ID')} });

}

}

Below you can find short snippets of usage. Be aware connect() should be called after registering the metrics.

// create producer

this.kafka = kafkaConfig.getConfig();

this.producer = this.kafka.producer();

// monitor before connect()

kafkaConfigService.monitorProducer(this.producer);

await this.producer.connect();

// create consumer

this.kafka = kafkaConfigService.getConfig();

this.consumer = this.kafka.consumer(config);

// monitor before connect()

kafkaConfigService.monitorConsumer(this.consumer);

await this.consumer.connect();

You want to use @DataJpaTest annotation whenever writing an Integration Test for JPA related components of your application like Entities or Repositories.

@DataJpaTest annotation configures:

• In-memory embedded database – behavior can be disabled with @AutoConfigureTestDatabase(replace = Replace.NONE)
• Scans and configures @Entity beans
• Scans and configures Spring Data Repositories
• Configures TestEntityManager
• Does not load other components like @Component, @Service, @Controller etc.

Every @DataJpaTest is transactional by default, after each test transaction is rolled back. You can use @Transactional annotation to customize this behavior.

When using @DataJpaTest you can access TestEntityManager, which contains subset of EntityManager methods that are useful for testing.

@Mock annotation comes from Mockito Framework which allows for easy Mock creation. This annotation is used by MockitoJUnitRunner, each field annotated with it will have Mock for specified class created. This annotation does not inject mocks into tested class on itself, to use injection you need to have target class annotated with @InjectMocks annotation.

@MockBean annotation comes from spring-boot-test, it creates Mockito Mock and also injects it into Application Context created by @SpringBootTest. All beans which refers to mocked class via @Autowired will get this mock injected instead of real class.

Main difference between @MockBean and @Mock annotation is that @MockBean creates mock and injects it into Application Context, while @Mock annotation only creates it, if you want to inject it, you can do it manually or with @InjectMocks annotation, however injection is being done to the class not whole Application Context.

You should use @WebMvcTest annotation when you want to write Integration Test that is focused on web layer of your application. @WebMvcTest approach will create ApplicationContext that contains only web components and omits any other components that are not part of web layer. Other components, if required for the test, can be mocked with usage of @MockBean annotation or delivered by @Configuration annotated class imported with usage of @Import annotation.

@WebMvcTest supports two cases:

1. Single Controller Auto-Configuration – annotate test by providing Controller class - @WebMvcTest(CityController.class)

1. Multiple (All found) Controllers Auto-Configuration – just annotate test with @WebMvcTest

@WebMvcTest annotation will auto-configure:

• Mock Mvc
• @Controller annotated class
• @ControllerAdvice annotated class
• @JsonComponent annotated class
• @Converter annotated class
• @GenericConverter annotated class
• @Filter annotated class
• @WebMvcConfigurer annotated class
• @HandlerMethodArgumentResolver annotated class

Integration Test by definition, should check interactions between few components of the system (at least two real, not-mocked components) to check if those components are delivering expected functionalities when working together. In each case when writing Integration Test you should decide how many components should interact in the test for it to be meaningful. Usually, you should decide on smallest possible amount of components that are enough to test specific functionality. Components that are not meaningful can be omitted, or mocked with usage of @MockBean annotation.

Web components tests (Controller Tests, Rest Controller Tests), if tested in Integration way, should be written in a way for test to make a HTTP Request and check HTTP Response. This kind of approach results in meaningful test, which delivers feedback that actually checks if component works correctly.

Spring Boot allows you to write Integration Tests for Web Components in two ways:

spring-boot-starter-test brings following dependencies:

• JUnit - Unit Testing for Java Applications
• Spring Test - Spring Framework Support for Testing
• Spring Boot Test - Utilities and Integration Test Support for Spring Boot
• AssertJ - Fluent Assertion Library
• Hamcrest - Matchers Library
• Mockito - Mocking Framework
• JSONassert - JSON Assertion Library
• JsonPath - XPath for JSON
• XMLUnit - Tools for XML verification

You can see list of all dependencies with versions for maven module by running following command:

mvn dependency:tree

[INFO] \- org.springframework.boot:spring-boot-starter-test:jar:2.1.4.RELEASE:test
[INFO]    +- org.springframework.boot:spring-boot-starter:jar:2.1.4.RELEASE:test
[INFO]    |  +- org.springframework.boot:spring-boot:jar:2.1.4.RELEASE:test
[INFO]    |  |  \- org.springframework:spring-context:jar:5.1.6.RELEASE:test
[INFO]    |  |     +- org.springframework:spring-aop:jar:5.1.6.RELEASE:test
[INFO]    |  |     +- org.springframework:spring-beans:jar:5.1.6.RELEASE:test
[INFO]    |  |     \- org.springframework:spring-expression:jar:5.1.6.RELEASE:test
[INFO]    |  +- org.springframework.boot:spring-boot-autoconfigure:jar:2.1.4.RELEASE:test
[INFO]    |  +- org.springframework.boot:spring-boot-starter-logging:jar:2.1.4.RELEASE:test
[INFO]    |  |  +- ch.qos.logback:logback-classic:jar:1.2.3:test
[INFO]    |  |  |  \- ch.qos.logback:logback-core:jar:1.2.3:test
[INFO]    |  |  +- org.apache.logging.log4j:log4j-to-slf4j:jar:2.11.2:test
[INFO]    |  |  |  \- org.apache.logging.log4j:log4j-api:jar:2.11.2:test
[INFO]    |  |  \- org.slf4j:jul-to-slf4j:jar:1.7.26:test
[INFO]    |  +- javax.annotation:javax.annotation-api:jar:1.3.2:test
[INFO]    |  \- org.yaml:snakeyaml:jar:1.23:test
[INFO]    +- org.springframework.boot:spring-boot-test:jar:2.1.4.RELEASE:test
[INFO]    +- org.springframework.boot:spring-boot-test-autoconfigure:jar:2.1.4.RELEASE:test
[INFO]    +- com.jayway.jsonpath:json-path:jar:2.4.0:test
[INFO]    |  +- net.minidev:json-smart:jar:2.3:test
[INFO]    |  |  \- net.minidev:accessors-smart:jar:1.2:test
[INFO]    |  |     \- org.ow2.asm:asm:jar:5.0.4:test
[INFO]    |  \- org.slf4j:slf4j-api:jar:1.7.26:test
[INFO]    +- junit:junit:jar:4.12:test
[INFO]    +- org.assertj:assertj-core:jar:3.11.1:test
[INFO]    +- org.mockito:mockito-core:jar:2.23.4:test
[INFO]    |  +- net.bytebuddy:byte-buddy:jar:1.9.12:test
[INFO]    |  +- net.bytebuddy:byte-buddy-agent:jar:1.9.12:test
[INFO]    |  \- org.objenesis:objenesis:jar:2.6:test
[INFO]    +- org.hamcrest:hamcrest-core:jar:1.3:test
[INFO]    +- org.hamcrest:hamcrest-library:jar:1.3:test
[INFO]    +- org.skyscreamer:jsonassert:jar:1.5.0:test
[INFO]    |  \- com.vaadin.external.google:android-json:jar:0.0.20131108.vaadin1:test
[INFO]    +- org.springframework:spring-core:jar:5.1.6.RELEASE:test
[INFO]    |  \- org.springframework:spring-jcl:jar:5.1.6.RELEASE:test
[INFO]    +- org.springframework:spring-test:jar:5.1.6.RELEASE:test
[INFO]    \- org.xmlunit:xmlunit-core:jar:2.6.2:test
[INFO]       \- javax.xml.bind:jaxb-api:jar:2.3.1:test
[INFO]          \- javax.activation:javax.activation-api:jar:1.2.0:test
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------