Build Components
Purpose
This section deploys the main components of the Node, including the IA Node, Management Node and Federators.
Stage 3 - Deploy IA Node Components
How to Complete This Stage
- Deploy Strimzi and Kafka, and apply a demonstration Access Control List
- Deploy the IA Node
- Deploy the Management Node
- Deploy demonstration Server and Client Federators
- Create a Trust Store and a Key Store for certificate management
3.1 Kafka deployment (Strimzi)
Deploy Strimzi for Kafka cluster management:
helm install my-strimzi-cluster-operator oci://quay.io/strimzi-helm/strimzi-kafka-operator --namespace kafka-operator --create-namespace \
--set watchAnyNamespace="true" \
--set resources.requests.cpu=50m \
--set resources.requests.memory=128Mi \
--set resources.limits.cpu=500m \
--set resources.limits.memory=256Mi
Deploy Kafka using the ia-node-kaka helper chart:
If you encounter errors when pulling and installing the chart directly, run the installation from the downloaded chart, adding the flag --server-side=false:
3.2 Configuration Kakfka ACLs
Configure all Kafka Access Control Lists (ACLs) in a single patch operation
kubectl patch kafkauser kafka-ia-node-user -n org-a --type='json' -p='[
{
"op": "replace",
"path": "/spec/authorization/acls/0/operations",
"value": ["Create", "Describe", "DescribeConfigs", "Read", "Write"]
},
{
"op": "replace",
"path": "/spec/authorization/acls/1/operations",
"value": ["Create", "Describe", "DescribeConfigs", "Read", "Write"]
},
{
"op": "add",
"path": "/spec/authorization/acls/-",
"value": {
"operations": ["Read"],
"resource": {
"type": "group",
"name": "JenaFusekiKafka-",
"patternType": "prefix"
}
}
},
{
"op": "add",
"path": "/spec/authorization/acls/-",
"value": {
"operations": ["Read"],
"resource": {
"type": "group",
"name": "federator-server",
"patternType": "literal"
}
}
},
{
"op": "add",
"path": "/spec/authorization/acls/-",
"value": {
"operations": ["Read"],
"resource": {
"type": "group",
"name": "management-node",
"patternType": "literal"
}
}
},
{
"op": "add",
"path": "/spec/authorization/acls/-",
"value": {
"operations": ["Create", "Describe", "Write", "Read"],
"resource": {
"type": "topic",
"name": "DEMOPRODUCERIANODE-",
"patternType": "prefix"
}
}
},
{
"op": "add",
"path": "/spec/authorization/acls/-",
"value": {
"operations": ["Describe", "DescribeConfigs"],
"resource": {
"type": "cluster",
"patternType": "literal"
}
}
},
{
"op": "add",
"path": "/spec/authorization/acls/-",
"value": {
"operations": ["Describe", "DescribeConfigs"],
"resource": {
"type": "topic",
"name": "DEMOPRODUCERIANODE-",
"patternType": "prefix"
}
}
}
]'
Confirm that the ACLs were added:
3.3 Deploy IA Node
Navigate to ~/src/helm-charts/charts/ia-node/templates. Open default-peerauthentication.yaml and set MTLS mode to "STRICT" or "PERMISSIVE" as required
Install the IA node:
helm -n org-a install ia-node ./charts/ia-node \
--set apps.api.configMap.data.DEPLOYED_DOMAIN="http://localhost" \
--set apps.api.configMap.data.OPENID_PROVIDER_URL="http://keycloak.keycloak.svc.cluster.local/realms/ianode/" \
--set apps.graph.configMap.data.JWKS_URL="http://keycloak.keycloak.svc.cluster.local/realms/ianode/protocol/openid-connect/certs" \
--set apps.graph.configMap.data.USER_ATTRIBUTES_URL="http://access-api.org-a.svc.cluster.local:8080/users/lookup/{user}" \
--set apps.graph.configMap.data.ATTRIBUTE_HIERARCHY_URL="http://access-api.org-a.svc.cluster.local:8080/hierarchies/lookup/{name}" \
--set apps.graph.enabled=true \
--set apps.api.deployment.resources.requests.cpu=25m \
--set apps.graph.statefulSet.resources.requests.cpu=25m \
--set apps.graph.statefulSet.resources.requests.memory=128Mi \
--set kafkaCluster.bootstrapServers="kafka-cluster-kafka-bootstrap.org-a.svc.cluster.local:9092" \
--set fusekiConfig.cqrsEnabled=false \
--set fusekiConfig.kafkaTopics.knowledge="DEMOPRODUCERIANODE-knowledge" \
--set fusekiConfig.kafkaTopics.ontology="DEMOPRODUCERIANODE-ontology.car.models" \
--set istio.virtualService.hosts[0]="localhost"
3.4 Deploy Management Node database
Deploy a Postgresql database to stored federation data, replacing the password with an appropriate value:
kubectl create namespace central-org
kubectl label namespace central-org istio-injection=enabled
helm repo add bitnami https://charts.bitnami.com/bitnami
helm upgrade --install management-postgres bitnami/postgresql -n central-org \
--set architecture=standalone \
--set auth.username=mn_user \
--set auth.password=<choose password> \
--set auth.database=management_node \
--set primary.persistence.enabled=true
3.5 Deploy the Management Node
helm upgrade --install management-node ./charts/management-node \
-n central-org \
--set app.datasource.secret.username=mn_user \
--set app.datasource.secret.password=${POSTGRESS_PASSWORD} \
--set app.datasource.secret.create=true \
--set app.datasource.url=jdbc:postgresql://management-postgres-postgresql.central-org.svc.cluster.local:5432/management_node \
--set image.repository=ghcr.io/national-digital-twin/management-node/management-node \
--set image.tag=1.0.1 \
--set app.ssl.enabled=false \
--set app.oauth2.resourceserver.jwt.issuerUri=http://keycloak.keycloak.svc.cluster.local/realms/management-node \
--set app.oauth2.resourceserver.jwt.jwkSetUri=http://keycloak.keycloak.svc.cluster.local/realms/management-node/protocol/openid-connect/certs \
--set app.oauth2.resourceserver.opaquetoken.introspectionUri=http://keycloak.keycloak.svc.cluster.local/realms/management-node/protocol/openid-connect/token/introspect \
--set app.oauth2.resourceserver.opaquetoken.clientSecret=${CLIENT_SECRET} \
--set app.oauth2.resourceserver.jwt.jwsAlgorithm=RS256
Retrieve DB Password:
PGPASSWORD=$(kubectl get secret management-postgres-postgresql -n central-org -o jsonpath='{.data.password}' | base64 -d)
3.6 Configure Management Node data
Execute the following statements
PGPASSWORD=$(kubectl get secret management-postgres-postgresql -n central-org -o jsonpath='{.data.password}' | base64 -d)
# Insert consumer
kubectl exec -n central-org management-postgres-postgresql-0 -- bash -c "PGPASSWORD=$PGPASSWORD psql -U mn_user -d management_node -c \"INSERT INTO mn.consumer (idp_client_id, name, org_id) VALUES ('management-node', 'IA-NODE-CONSUMER-1', 1);\""
# Insert producer - pointing to federator-server (not management-node)
kubectl exec -n central-org management-postgres-postgresql-0 -- bash -c "PGPASSWORD=$PGPASSWORD psql -U mn_user -d management_node -c \"INSERT INTO mn.producer (idp_client_id, name, description, host, port, active, tls, org_id) VALUES ('management-node', 'DEMO-PRODUCER-IA-NODE', 'Demo producer for IA Node', 'federator-server.org-b.svc.cluster.local', 9001, true, false, 1);\""
# Add product_type column (required for federator-client)
kubectl exec -n central-org management-postgres-postgresql-0 -- bash -c "PGPASSWORD=$PGPASSWORD psql -U mn_user -d management_node -c \"ALTER TABLE mn.product ADD COLUMN IF NOT EXISTS product_type VARCHAR(50);\""
# Insert products with product_type
kubectl exec -n central-org management-postgres-postgresql-0 -- bash -c "PGPASSWORD=$PGPASSWORD psql -U mn_user -d management_node -c \"INSERT INTO mn.product (name, topic, product_type, producer_id) SELECT 'CarModelsKnowledge', 'knowledge', 'KNOWLEDGE', id FROM mn.producer WHERE idp_client_id = 'management-node';\""
kubectl exec -n central-org management-postgres-postgresql-0 -- bash -c "PGPASSWORD=$PGPASSWORD psql -U mn_user -d management_node -c \"INSERT INTO mn.product (name, topic, product_type, producer_id) SELECT 'CarModelsOntology', 'ontology.car.models', 'ONTOLOGY', id FROM mn.producer WHERE idp_client_id = 'management-node';\""
# Link the products to the built-in `topic` product type (id=1)
kubectl exec -n central-org management-postgres-postgresql-0 -- bash -c "PGPASSWORD=$PGPASSWORD psql -U mn_user -d management_node -c \"ALTER TABLE mn.product ADD COLUMN IF NOT EXISTS product_type_id INT;\""
kubectl exec -n central-org management-postgres-postgresql-0 -- bash -c "PGPASSWORD=$PGPASSWORD psql -U mn_user -d management_node -c \"UPDATE mn.product SET product_type_id = 1 WHERE name IN ('CarModelsKnowledge', 'CarModelsOntology');\""
# Link products to consumer
kubectl exec -n central-org management-postgres-postgresql-0 -- bash -c "PGPASSWORD=$PGPASSWORD psql -U mn_user -d management_node -c \"INSERT INTO mn.product_consumer (product_id, consumer_id, granted_ts, validity) SELECT p.id, c.id, '2025-11-06 00:00:00', 365 FROM mn.product p, mn.consumer c WHERE p.name = 'CarModelsKnowledge' AND c.idp_client_id = 'management-node';\""
kubectl exec -n central-org management-postgres-postgresql-0 -- bash -c "PGPASSWORD=$PGPASSWORD psql -U mn_user -d management_node -c \"INSERT INTO mn.product_consumer (product_id, consumer_id, granted_ts, validity) SELECT p.id, c.id, '2025-11-06 00:00:00', 365 FROM mn.product p, mn.consumer c WHERE p.name = 'CarModelsOntology' AND c.idp_client_id = 'management-node';\""
# Update the polling schedule to run every 4 hours
kubectl exec -n central-org management-postgres-postgresql-0 -- bash -c "PGPASSWORD=$PGPASSWORD psql -U mn_user -d management_node -c \"alter table mn.product_consumer add column schedule_type VARCHAR(100);\""
kubectl exec -n central-org management-postgres-postgresql-0 -- bash -c "PGPASSWORD=$PGPASSWORD psql -U mn_user -d management_node -c \"alter table mn.product_consumer add column schedule_expression VARCHAR(255);\""
kubectl exec -n central-org management-postgres-postgresql-0 -- bash -c "PGPASSWORD=$PGPASSWORD psql -U mn_user -d management_node -c \"UPDATE mn.product_consumer SET schedule_type = 'cron', schedule_expression = '0 */4 * * *' WHERE id IN (4, 5);\""
echo "Configuration added successfully"
Verify the changes by listing existing consumers and producers:
kubectl exec -n central-org management-postgres-postgresql-0 -- bash -c "PGPASSWORD=$PGPASSWORD psql -U mn_user -d management_node -c \"SELECT id, idp_client_id, name FROM mn.consumer;\""
kubectl exec -n central-org management-postgres-postgresql-0 -- bash -c "PGPASSWORD=$PGPASSWORD psql -U mn_user -d management_node -c \"SELECT id, idp_client_id, name, host FROM mn.producer;\""
Verify that the configuration has been inserted. You should see one record for each topic:
PGPASSWORD=$(kubectl get secret management-postgres-postgresql -n central-org -o jsonpath='{.data.password}' | base64 -d)
kubectl exec -n central-org management-postgres-postgresql-0 -- bash -c "PGPASSWORD=$PGPASSWORD psql -U mn_user -d management_node -c \"
SELECT
c.idp_client_id as consumer,
p.name as product,
p.topic,
pr.name as producer_name,
pr.host as producer_host,
pr.port as producer_port,
COUNT(pca.id) as filter_attribute_count
FROM mn.consumer c
JOIN mn.product_consumer pc ON c.id = pc.consumer_id
JOIN mn.product p ON pc.product_id = p.id
JOIN mn.producer pr ON p.producer_id = pr.id
LEFT JOIN mn.product_consumer_attribute pca ON pc.id = pca.product_consumer_id
WHERE c.idp_client_id = 'management-node'
GROUP BY c.idp_client_id, p.name, p.topic, pr.name, pr.host, pr.port;
\""
3.7 Create certificate stores
Create truststore:
keytool -genkeypair -alias dummy -keystore /tmp/truststore.jks \
-storepass changeit -keypass changeit -dname "CN=dummy" -keyalg RSA
keytool -delete -alias dummy -keystore /tmp/truststore.jks -storepass changeit
Create Kubernetes secret
kubectl -n org-a create secret generic federator-client-certs \
--from-file=truststore.jks=/tmp/truststore.jks
Copy Keycloak client secret from keycloak namespace:
kubectl get secret keycloak-management-node-client -n keycloak -o yaml | sed 's/namespace: keycloak/namespace: org-a/' | kubectl apply -f -
kubectl get secret keycloak-management-node-client -n keycloak -o yaml | sed 's/namespace: keycloak/namespace: org-b/' | kubectl apply -f -
3.8 Deploy Federator Server
If necessary, create and label the namespace for the server:
Copy Kafka user secret (JAAS) for federator-server
kubectl get secret kafka-ia-node-user -n org-a -o json \
| jq 'del(
.metadata.namespace,
.metadata.resourceVersion,
.metadata.uid,
.metadata.creationTimestamp,
.metadata.managedFields,
.metadata.ownerReferences,
.metadata.labels
)
| .metadata.namespace="org-b"' \
| kubectl apply -f -
Copy MongoDB password secret (if graph-server will be deployed in org-a)
kubectl get secret ia-node-user-password -n org-a -o yaml | sed 's/namespace: org-a/namespace: org-b/' | kubectl apply -f -
Copy Redis password
kubectl get secret oauth2-proxy-redis -n org-a -o yaml | sed 's/namespace: org-a/namespace: org-b/' | kubectl apply -f -
Create valid empty truststore for federator-server
keytool -genkeypair -alias dummy -keystore /tmp/truststore.jks \
-storepass changeit -keypass changeit -dname "CN=dummy" -keyalg RSA -keysize 2048
keytool -delete -alias dummy -keystore /tmp/truststore.jks -storepass changeit
Create valid empty keystore for federator-server (required even when mTLS is disabled)
keytool -genkeypair -alias dummy -keystore /tmp/keystore.p12 -storetype PKCS12 \
-storepass changeit -keypass changeit -dname "CN=dummy" -keyalg RSA -keysize 2048
Create Kubernetes secret with both truststore and keystore
kubectl -n org-b create secret generic federator-server-certs \
--from-file=truststore.jks=/tmp/truststore.jks \
--from-file=keystore.p12=/tmp/keystore.p12
Deploy the Federator Server:
helm upgrade --install federator-server ./charts/federator --namespace org-b --values ./charts/federator/examples/values-server-example.yaml
Checkpoint
Before proceeding, confirm:
- Kafka is deployed and ACLs are applied
- IA Node is running
- Management Node and database are deployed
- Required configuration data is inserted
- Certificate secrets exist
- Federator Server is deployed successfully
You are now ready to test communication between the Federator Server and Client.
Next step
Proceed to: Run and Validate