Transparency

Overview

The Transparency service allows you to easily gather all the personal data of a data subject stored across all YaaS services and systems. The service serves the data subject's right to have access, information about, and portability for their personal data, and enables projects to easily gather that data across all YaaS services.

The Transparency service, in response to the inquiry, contacts the data privacy registry for information about which services store any personal data of a data subject. Then the Transparency service starts the events chain to gather data from all applicable services.


API Reference

/inquiries

An inquiry represents a demand to create a report that contains information about the personal data stored for a data subject. Data subjects are the people who requested their own personal data.

/inquiries

post

Creates a new inquiry for a report that contains information about the personal data stored for a data subject.

Security / Access Control:
To access this method, request the hybris.transparency_manage scope with the access token.

get

This call gets an inquiries list for a tenant. You can filter the list by the data subject type and data subject identifier.

Security / Access Control:
To access this method, request the hybris.transparency_manage scope with the access token.

/inquiries/{id}

/inquiries/{id}

get

This call gets an inquiry for a specific inquiry ID.

Security / Access Control:
To access this method, request the hybris.transparency_manage scope with the access token.

/inquiries/{id}/report

/inquiries/{id}/report

get

This request gets a report for the specified inquiry ID.

Security / Access Control:
To access this method, request the hybris.transparency_manage scope in the access token request.

/inquiries/{id}/responses

/inquiries/{id}/responses

post

This request stores the response from a service that holds personal data. Find documentation describing which headers to set in specific flows on the Dev Portal in the API documentation for the Transparency service.

Security / Access Control:
To access this method, request the hybris.transparency_notify scope in your request for an access token.


Events

For more information about events, see the PubSub service documentation.

The topic owner client is: personaldata.transparency

The Transparency service broadcasts the following event:

eventType - transparency-request

TypeDescriptionSchemaPayload Example
transparency-requestThe Transparency service created a inquiry for personal data and requires a response.
{
          "$schema": "http://json-schema.org/draft-04/schema#",
          "title": "transparency-request",
          "description": "The submitted push event.",
          "type": "object",
          "properties": {
            "dataSubjectType": {
              "type": "string",
              "description": "Data subject type, such as yaas-account or caas-customer"
            },
            "dataSubjectId": {
              "type": "string",
              "description": "Data subject identifier"
            },
            "requestId": {
              "type": "string",
              "description": "The request ID used to correlate the response to a transparency inquiry."
            }
          },
          "required": [
            "dataSubjectType",
            "dataSubjectId",
            "requestId"
          ]
}
{
         "dataSubjectType": "caas-customer",
         "dataSubjectId": "fe512f2b-9d13-496c-b8d3-4c13c42e43cd",
         "requestId": "c4a8e790-9d10-4aa7-ba7c-e37472ecba30"
}

Please bear in mind that the payload for an event is stored as a JSON encoded string and requires an extra decoding run.


Considerations

The system does not return the personal data immediately after you issue an inquiry to the Transparency service. The inquiry triggers an events chain through the PubSub service. All services that receive the transparency event are obliged to respond to the Transparency service. Those that contain a subject's personal data respond to the Transparency service with the data.


Personal data inquiry

Request a report with a personal data

To request a report containing all of a data subject's personal data, perform a POST request on the /inquiries endpoint. In the body of the request, provide:

  • the dataSubjectType - the type of the personal data subject
  • the dataSubjectId - the identifier of a personal data subject

Example:

{
  "dataSubjectType": "caas-customer",
  "dataSubjectId": "fe512f2b-9d13-496c-b8d3-4c13c42e43cd"
}

The Transparency service immediately responds with an inquiry ID and starts the processing in the background.

This inquiry demands that the service create a report which contains information about all of the subject's personal data stored across all YaaS services.

The service communicates with the data privacy registry and the PubSub service. The Transparency service gets information from the registry as to which services are registered as containing personal data, indicated when the personalData field value is set with the markets codes. Then the Transparency service checks subscriptions, and based on both sources, it determines which services to await responses from. The Transparency service broadcasts the transparency event through the PubSub service and all the services that receive the transparency event process it and respond directly to the Transparency service.

With each response from the service, the Transparency service updates the status of a transparency inquiry. If not all of the concerned services respond to the event in time, the service sets the inquiry status to PARTIALLY_FINISHED. The Transparency service sets the inquiry status to FINISHED when it receives positive responses from all the services concerned.

List transparency inquiries

Perform a GET request on the /inquiries endpoint to list all the transparency inquiries in the scope of the requester.

Example:

[
  {
    "id": "c4a8e790-9d10-4aa7-ba7c-e37472ecba30",
    "dataSubjectType": "caas-customer",
    "dataSubjectId": "fe512f2b-9d13-496c-b8d3-4c13c42e43cd",
    "status": "FINISHED",
    "createdAt": "2017-05-29T06:29:45.526Z",
    "modifiedAt": "2017-05-29T06:30:00.000Z",
    "finishedAt": "2017-05-29T06:33:00.000Z",
    "reportAvailable": true
  },
  {
    "id": "3b79de3f-899d-43ae-b569-ccfefbd87c3d",
    "dataSubjectType": "caas-customer",
    "dataSubjectId": "f846d43d-910f-4832-87de-4cf6305bd51a",
    "status": "IN_PROGRESS",
    "createdAt": "2017-05-30T09:19:26.354Z",
    "modifiedAt": "2017-05-30T09:19:26.354Z",
    "reportAvailable": false
  }
]

Filter your inquiries

You can query your inquiries using one or both of the dataSubjectId and dataSubjectType parameters:

/inquiries?dataSubjectType=x&dataSubjectId=y

Get details of a transparency inquiry

To get a particular inquiry by ID, perform a GET request on the /inquiries/{id} endpoint. In response, you receive a set of information containing all the details of the inquiry.

The status field contains the status of the transparency inquiry. Possible values are:

  • in_progress - The Transparency service is waiting for the inquiry to finish and the report to generate.
  • partially_finished - The report is ready for some services, but others failed to respond in time. The Transparency service rejects further responses from the services. The data subject can retrieve the partial report.
  • finished - The report is ready and available under the /inquiries/{id}/report endpoint. The data subject can retrieve the report.

The services field contains basic details of responses from all services that contain personal data about the data subject in question:

  • serviceBasePath - the base path of the responding service
  • serviceRegion - the region for which the service keeps a data subject's personal data
  • status - the status of the response for the particular service, which allows these values:
    • not_responded - the service did not respond to the Transparency service yet
    • in_progress - the Transparency service is in the process of responding, in the case of a multi-element response
    • finished - the service responded
  • hasData - an optional field, indicating whether the service stores personal data for a given data subject

An example response follows:

{
  "id": "c4a8e790-9d10-4aa7-ba7c-e37472ecba30",
  "dataSubjectType": "caas-customer",
  "dataSubjectId": "fe512f2b-9d13-496c-b8d3-4c13c42e43cd",
  "status": "FINISHED",
  "createdAt": "2017-05-29T06:29:45.526Z",
  "modifiedAt": "2017-05-29T06:30:00.000Z",
  "finishedAt": "2017-05-29T06:33:00.000Z",
  "services": [
    {
      "serviceBasePath": "documentService",
      "serviceRegion": "us",
      "status": "in_progress",
      "hasData": true
    },
    {
      "serviceBasePath": "mediaService",
      "serviceRegion": "eu",
      "status": "finished",
      "hasData": false
    },
    {
      "serviceBasePath": "emailService",
      "serviceRegion": "eu",
      "status": "not_responded"
    }
  ],
  "reportAvailable": false
}

Get the report file

To get a report with all the personal data stored in services across YaaS, perform a GET request on the /inquiries/{id}/report endpoint.

The system returns the report_.zipfile as the result of the data inquiry. This file contains directories for each service. Each service directory consists of directories for each region in which the service stores personal data of a data subject in question. Each region directory contains a full set of files that contain personal data. The following shows an example structure:

├── report_<reqID>.zip
│   ├── hybris-shipment-1
│   │   ├── eu
│   │   │   ├── avatar.png
│   │   │   ├── data-content.json
│   │   │   └── data-content.txt
│   │   └── us
│   │       └── data-content.txt
│   ├── hybris-shoes2sell-v1
│   │   └── us
│   │       └── data-content.txt
│   └── index.html

Security

All requests to inquiry-related endpoints require an OAuth2 token with the scope hybris.transparency_manage.

There are two distinct flows when accessing the Transparency service API during a transparency inquiry: The YaaS account flow and the project context flow.

YaaS account flow

This flow is dedicated when data subject has a YaaS account and inquires about which personal data the system holds on the platform level. This flow does not address inquires in the context of a project (tenant) or organization.

All requests require an Oauth2 token without a project or organization scope. Request the scope hybris.no_tenant.

Additionally, when creating an inquiry, the value of the hybris-user-id from the token must match the value of the dataSubjectId in the request body, and you must set the dataSubjectType to yaas-account.

Project context flow

This flow is dedicated for Data Privacy Officers making an inquiry within the context of a specific project. In this flow, all inquires are in the context of a single project.

All requests require an OAuth2 token in the context of a project (tenant), and you must set the scope hybris.tenant. For example, hybris.tenant=shoes2sell.


Integrate your service

Subscriptions

The Transparency service supports different flows for each of the two data subject types: yaas-account and project related type.

YaaS account flow

The first flow serves the yaas-account data subject type.

The Transparency service checks the list of services registered with a personalData flag in the data privacy registry. The Transparency service checks the list of all active subscriptions to the personaldata tenant, compares both lists, and then creates an internal list of those services that appear in both lists. The Transparency service sends an event to the /personaldata.transparency/transparency-request topic for the personaldata tenant and waits for responses from the services appearing only in the internal list.

To receive transparency events for the yaas-account flow, your service must register with the personalData flag, and the personaldata project must subscribe to your service.

Project context flow

The second flow serves the data subject type connected with a project. For example, caas-customer. The Transparency service checks the list of services registered with a personalData flag in the data privacy registry. The Transparency service checks the list of all active subscriptions to the given tenant, compares both lists, and then creates an internal list of those services that appear in both lists. The Transparency service sends an event to the /personaldata.transparency/transparency-request topic for the given tenant and waits for responses from the services appearing only in the internal list.

Respond to the transparency event

When the transparency event reaches your service, you must respond by performing a POST request on the /inquiries/{id}/responses/{serviceId} with one of the following statuses:

  • no-data - The service does not store any personal data for a given data subject.
  • data - The service responds with the data related to the given data subject.

If your service stores the data for a data subject, perform a POST request on the /inquiries/{id}/responses/{serviceId} with the data.

You can send the data in multiple responses. The last response must mark its completion. If you send the data in a single response, the response must also mark its completion.

With your POST request, provide the following headers:

  • hybris-transparency-service-base-path - the base path of the service, for example /hybris/shipment/v1

  • hybris-transparency-service-region - the region in which the service is deployed, for example EU

  • hybris-transparency-file-name - the name for the data sent in the body of your request

  • hybris-transparency-response-completed - A Boolean flag, when set to true, indicates that the service identified in the hybris-transparency-service-base-path header and deployed in the region identified in the hybris-transparency-service-region header completed uploading the responses.

You must make sure that the last response from the service for a given region, which is marked as complete, reaches the Transparency service last. When the Transparency service receives the response with the hybris-transparency-response-completed header set to true, it does not accept any further responses.
  • hybris-transparency-response-no-data - specifies whether the service has any data for a given inquiry; if the service does not contain relevant data, set the value to true

Be advised that there is a limited time window to provide a complete response to the inquiry.

Currently the time limit is set to one hour, which is pending since the Transparency service records the inquiry. This time is very close, on the order of milliseconds, to the time recorded in the createdAt field in the PubSub event.


Security

Data security

The Transparency service handles personal data that services provide as part of the transparency process. Data is encrypted when it enters the Transparency service and is always stored in an encrypted form. Data is decrypted when returned, in the form of a report in a ZIP archive.

The generation of a separate encryption key for each inquiry provides forward secrecy. The key in question is stored in database encrypted using master key.

Authenticated encryption protects data against ciphertext manipulation.

Personal data

The Transparency service does not retain the personal data that other services provide. The service only temporarily handles personal data provided as part of the transparency inquiry process, and the data is subject to obligatory and automatic retention.

The only personal information persisted is the data subject ID combined with the data subject type.

Backups

Operational data from the Transparency service is backed up automatically. Backups are encrypted locally before exposure to communication channels and storage.

Responses from services with personal data are not backed up.

Data residence and retention

Transparency inquiries are retained indefinitely and are accessible to an operator.

The retention period for personal data is four days after the Transparency service receives the data. The service sets the retention duration independently for each file.

The transparency report has a residence time of 48 hours after it is generated and the service retains it for four days in storage.

Backups are retained for 12 days.


Glossary

TermDescription
data privacy registryA platform component responsible for keeping track of services that contain personal data.
data subjectThe entity identified by object type and object ID.
personal dataAll the data that may lead to the identification of an individual, such as cultural or economic identity.


  • Send feedback

    If you find any information that is unclear or incorrect, please let us know so that we can improve the Dev Portal content.

  • Get Help

    Use our private help channel. Receive updates over email and contact our specialists directly.

  • hybris Experts

    If you need more information about this topic, visit hybris Experts to post your own question and interact with our community and experts.