Validators

While transformers can help auto-heal your data and bring consistency, they don't cover all cases. What if the email provided is invalid or the last name is blank? Validators have been designed to address this problem and provide feedback to your end-users, so they can fix these errors.

Let's see how we can extend our previous example to make sure that both email and last name are required. We will be using pipe syntax, like in transformers.

 1[
 2  {
 3    "label": "First name",
 4    "transformers": "trim|capitalize"
 5  },
 6  {
 7    "label": "Last name",
 8    "transformers": "trim|capitalize",
 9    "validators": "required" 
10  },
11  {
12    "label": "Phone",
13    "transformers": "trim"
14  },
15  {
16    "label": "Email",
17    "transformers": "trim|lowercase",
18    "validators": "required" 
19  },
20  {
21    "label": "Country",
22    "transformers": "trim|uppercase"
23  }
24]

#Chaining validators

Validators can be chained using the pipe syntax. Let's say we want to make sure that the email provided is valid. We can use the email validator.

 1[
 2  {
 3    "label": "First name",
 4    "transformers": "trim|capitalize"
 5  },
 6  {
 7    "label": "Last name",
 8    "transformers": "trim|capitalize",
 9    "validators": "required"
10  },
11  {
12    "label": "Phone",
13    "transformers": "trim"
14  },
15  {
16    "label": "Email",
17    "transformers": "trim|lowercase",
18    "validators": "required|email" 
19  },
20  {
21    "label": "Country",
22    "transformers": "trim|uppercase"
23  }
24]

#Available validators

#Custom Validators

On top of the built-in validators, you can always build your own validators so that you can report data errors and issues according to your requirements. To do this you will need to name your validator and associate it with a callback to be called every time it is used. You can add as many validators as necessary.

 1{
 2  "validators": {
 3    "my-validator": function(record, field) {
 4      ...
 5    },
 6    "my-validator-with-single-arg": function(record, field, arg1) {
 7      ...
 8    },
 9    "my-validator-with-multiple-args": function(record, field, arg1, arg2, arg3) {
10      ...
11    }
12  }
13}

Let's see how we can add a validator called phone to validate phone numbers using the google-libphonenumber library. In case something is wrong, we need to return the error message. Otherwise, we can return true to indicate that the provided value passes the validation. Please note that you can not overwrite a validator that already exists and you will get an error if you try to do that.

 1{
 2  "validators": {
 3    "phone": (record, key) => {
 4      const value = record.get(key);
 5      if (!value || value === '') {
 6        return true;
 7      }
 8 
 9      try {
10        const parsed = PhoneNumberUtil.getInstance().parseAndKeepRawInput(value);
11        if (!PhoneNumberUtil.getInstance().isValidNumber(parsed)) {
12          return 'Must be a valid phone number.';
13        }
14      } catch (e) {
15        return 'Must be a valid phone number.';
16      }
17 
18      return true;
19    }
20  }
21};

That's all! You can use the pipe syntax to use the new validator.

 1{
 2  "fields": [
 3    {
 4      "label": "First name",
 5      "transformers": "trim"
 6    },
 7    {
 8      "label": "Last name",
 9      "transformers": "trim"
10    },
11    {
12      "label": "Phone",
13      "transformers": "trim",
14      "validators": "phone"
15    },
16    {
17      "label": "Email",
18      "transformers": "trim"
19    }
20  ]
21}

#Passing additional arguments

Validators can be made more dynamic by accepting additional arguments. Let's enhance our example to accept a country code as the first argument.

 1{
 2  "validators": {
 3    "phone": (record, key, defaultCountry) => {
 4      const value = record.get(key);
 5      if (!value || value === '') {
 6        return true;
 7      }
 8 
 9      try {
10        const parsed = defaultCountry
11          ? PhoneNumberUtil.getInstance().parseAndKeepRawInput(value, defaultCountry)
12          : PhoneNumberUtil.getInstance().parseAndKeepRawInput(value);
13        if (!PhoneNumberUtil.getInstance().isValidNumber(parsed)) {
14          return 'Must be a valid phone number.';
15        }
16      } catch (e) {
17        return 'Must be a valid phone number.';
18      }
19 
20      return true;
21    }
22  }
23};

#Validators with Context

By default, validators validate one record at a time. However, there are scenarios where you need to validate a field across all records in the dataset. For example, checking if a value is unique across all rows or validating relationships between records.

Context validators receive all records as an array instead of a single record, allowing cross-record validation. It is required to add errors directly to individual records using record.addError. Please note that the built-in unique validator is implemented as a context validator since it needs to compare values across all records.

#Defining Custom Context Validators

Context validators must be defined in a separate withContext object in your configuration. Here is how you can define a custom context validator called valid-manager that checks if the "Manager ID" field references a valid employee in the dataset:

 1{
 2  "validators": {
 3    // Regular validators go here
 4    "phone": (record, key) => {
 5      // ... regular validator logic
 6      return true; // or error message
 7    },
 8    "withContext": {
 9      "valid-manager": (context, key) => {
10      const bag = 'valid-manager'; // Error namespace
11 
12      // First pass: collect all employee IDs and clear previous errors
13      const employeeIds = new Set();
14      for (const record of context) {
15        record.clearErrors(bag);
16        const empId = record.get('Employee ID');
17        if (empId) {
18          employeeIds.add(empId);
19        }
20      }
21 
22      // Second pass: validate manager IDs
23      for (const record of context) {
24        const managerId = record.get(key);
25 
26        if (managerId === '') {
27          continue; // Skip if no manager
28        }
29 
30        if (!employeeIds.has(managerId)) {
31          record.addError(key, 'Manager ID must reference a valid employee.', bag);
32        }
33      }
34    }
35  }
36}

Understanding error bags: The third parameter in addError() is called a "bag" - it's an error namespace that allows you to group and clear errors by validator. This is crucial for context validators because when data changes and validation re-runs, you need to clear only the errors from that specific validator without affecting errors from other validators. Always use record.clearErrors(bag) at the start of your context validator to prevent stale errors.

You can then use the context validator just like any other validator:

1{
2  "fields": [
3    {
4      "label": "Manager ID",
5      "transformers": "trim|lowercase",
6      "validators": "required|valid-manager"
7    }
8  ]
9}

Please note that context validators are more powerful but also more complex than regular validators. They require careful handling of errors and efficient algorithms to avoid performance issues, especially with large datasets.

#Important Notes

• Context validators have access to the entire dataset, so they run once for all records rather than once per record
• Context validators can access any field from any record, not just the field being validated
• Always use record.addError(key, message, bag) with a bag parameter to namespace your errors
• Always call record.clearErrors(bag) before adding new errors to avoid stale validation results
• Use Map or Set to track seen values and their associated records for better performance: O(n) vs O(n²) with nested loops

#Data Providers

Validators are great when you want to validate data synchronously. However, there are cases where you want to fetch data asynchronously from your API. For that you will need to use data providers.

In the next step we are going to extend the above example to push the normalized data to our API using webhooks.

Have a look at webhooks →