Articles tagged with 'automated testing'

TrialGrid Version 91 - Safer Deletion, User Properties and Organization Roles

Version 91 makes deletion both easier and safer. Deleting things is more direct than before, and more of what you delete can now be restored. The release also introduces a new User custom property type, finer control over what our AI Agents are told, and Organization Roles.

For more information see the release notes for Version 91.

Deleting With Confidence

Deletion is now easier where it used to get in the way. Fields can be deleted from the Fields list, individually or in bulk. Any object can be deleted from its own editor using the Delete option in the dropdown beside Save. Deleting a Field in the Form Editor no longer refuses when other objects depend on it. Instead, it warns you which Edit Checks, Derivations and Lab Settings will be removed with it, and they are removed when the Form is saved.

It is also safer. Deleting a Project, Draft or URL now requires you to type DELETE to confirm, so a mis-click can no longer destroy work. If something is deleted by mistake, more of it can be recovered. Deleted Drafts appear on a new Deleted tab of the Project's Drafts list, showing who deleted them and when they will be removed permanently, and they can be restored from there. Deleted Test Cases have their own Deleted page in the Test Case list, where you can select any number and restore them in the background. A restored Test Case does not get back its labels, comments, ticket links or Test Run history.

A Custom Property for People

Custom Properties can now be of type User. Setting a value opens a picker of names and email addresses, and in listings the chosen person is shown with their avatar. Actions can set a User property to whoever runs the action by using the expression user, which makes fields such as "Reviewed by" or "Owner" simple to fill in as part of a workflow.

Telling Agents What Your Data Means

Custom Properties can now carry their own agent instruction. It is sent to the AI alongside the property's value, so the model is told what that particular property means instead of relying on a single instruction for the whole object type. To see the whole picture, Object Definitions have a new Agent Instructions tab. It shows the object type's own instruction and, for each property, whether the value is sent and what guidance goes with it. Anyone who can see the URL can read it, and editing needs the Manage Agent Configuration permission.

The Bulk Edit Check Creator's action hooks now export an outcome variable naming the hook that fired. This lets you select one action under several hooks, for example with the precondition agent("outcome") in ["custom_function_required", "build_failed"].

Organization Roles

Organization-level permissions now work the same way as Project permissions, through named roles. The new Organization Roles page is where they are defined and viewed, and each role decides what its holders can do. Holding "Administer Permissions" no longer grants every other permission, and a new "Can manage Icons" permission controls who can change Organization Icons. The background to this change is covered in Organization Roles.

Automated Testing

When you register a Medidata Rave URL, you now choose whether it is Rave Classic (direct or via iMedidata) or Rave EDC (the old or new user interface). TrialGrid no longer infers this from the Rave version. Registering a URL again resets its user access, so access is checked before the next run, and any Test Case Runs in progress on it are cancelled once you confirm. A URL already on the new Rave EDC interface keeps its access when it is registered as the new interface again.

When an expected email is not received, TrialGrid now lists the emails your organization did receive during the Scenario, with their date, time and subject, and highlights the differences between the expected text and the closest match. Most missing emails turn out to be near misses, and now you can see why.

Behind the scenes, a test run worker that is being retired is now returned to service when demand picks up again, rather than sitting idle while TrialGrid starts a replacement.

Veeva

On Veeva vaults at release 26R2 or later, test runs open forms directly using Veeva EDC's stable form links, and each form screenshot in the results links to that form in Veeva EDC. The Veeva Vaults list shows which Veeva release each vault is on. The release is read automatically from the vault when it is registered and whenever test cases run.

Settings Export

The URL settings export now includes a tab for Object Definition layouts, showing each layout definition and the roles it is assigned to.

This post was auto-generated by a LLM.

TrialGrid Version 90 - Organization-wide Roles, Smarter Actions and More Reliable Test Runs

Version 90 brings Project Roles up to the organization level, so they are defined once rather than once per URL. Actions can now calculate the values they set. Automated test runs check their own work more carefully, and do it faster. Veeva test generation reaches rules that span linked forms, and Document Templates can now run from the object lists you already work in.

For more information see the release notes for Version 90.

Project Roles, Defined Once

Until now, each URL in an organization had its own set of Project Role definitions. If you have more than a handful of URLs, keeping those sets consistent was tedious and easy to get wrong. In Version 90, Project Roles are defined and managed at the organization level and shared across all of an organization's URLs. Organization administrators manage them from the organization navigation, and a new bulk merge folds duplicate roles together. Role descriptions can now be edited too. We explained the thinking behind this change in Project Roles consolidated within an Organization.

Actions That Calculate

Actions could already collect a custom property value from the user or clear it. Now they can set a property to a fixed value, or to the result of an expression evaluated when the action runs. For example, a due date can be set to today() + days(10), a property can be set to the acting user, or one property can be copied from another on the same object. If an expression or a data collection step comes out empty, the property is cleared, just as Clear Property does. Action preconditions and standards rule logic also gain int() and float() conversions, and HTML custom properties collected by an action are now entered through a rich-text editor.

Custom Properties that belong to the built-in definition of an object type are now protected. They can no longer be edited, deleted or copied, so the properties TrialGrid relies on stay the way TrialGrid expects them.

Working Together in List Views

Spreadsheet views now have an Assignments column. It shows who is assigned to each object and lets you assign users to many objects at once. Columns for Labels, Standards, Test Cases and Custom Properties can now be chosen in the column picker even when they have no data yet, and every column you show is included in CSV, Excel, PDF, clipboard and print exports.

Document Templates Where You Need Them

Document Template reports can now be run directly from Draft object listings such as Forms, Fields and Custom Objects. The report receives the objects you have selected, plus those that match your current search and filters, so the list you see becomes the input to the report.

The AI tag inside Document Templates has also improved. It supports effort levels, and if output is truncated it now says so rather than quietly cutting it off. The new ai_json_list and ai_json_object functions return structured data, so the template controls the layout instead of the model.

Reusing Your Standard Library

A new Import Applicable Library Objects command on the Draft home page searches the Draft's Standard Library for Matrices, Edit Checks and Derivations whose Folders, Forms and Fields already exist in the Draft. You choose which ones to bring across, and any Custom Functions they depend on come with them. When you build a study from a standard, this fills in the logic that already fits the forms you have.

Test Runs That Check Their Own Work

Data entry steps now confirm that the values TrialGrid entered were actually stored. If a save stored nothing, the step retries it, and if the stored values differ from what was entered, the step fails and says so. Data entry is also faster. TrialGrid now reads Rave's own response to each save to see whether the Subject's Folders or Forms changed, and re-reads the Subject only when they did. When a step looking for a Folder name doesn't find it, a screenshot of the subject's folders is now captured so you can see what was there instead.

Email steps are easier to write and easier to trust. Subject and message matching now ignores differences in whitespace, line breaks and case. Expected text can include placeholders such as {subject_name} and {testcase_name}. The email forwarded as evidence now records the Scenario name, the line number of the email step and the step text, including the expected message, so each piece of evidence can be traced back to the step that produced it.

iMedidata service accounts are now held per organization, and TrialGrid rotates their stored passwords automatically on a schedule.

Veeva

The Veeva Test Case Advisor now generates test cases for rules that read an item on a linked form. The generated test enters data on both forms and links them with the "I link form" step, including when the rule reaches the linked form through a Form Link item. Setting up a run is simpler too: the Test Set Run wizard now lists the studies and sites the Vault account can reach, so you no longer have to type them in.

Veeva Lab Codelists and Lab Unit Codelists are now imported as object types of their own. A Lab codelist and a study codelist with the same name can therefore both be held in the same draft, and items that reference a Lab codelist resolve to it through new Lab Codelist and Lab Unit Codelist properties. Existing drafts are converted automatically. Lab codelists that were previously skipped because of a name collision need a re-import from the SDS.

AI Usage Limits

AI usage is now limited by cost rather than by tokens. Each organization has a monthly AI budget and each user has a daily cost limit. The AI usage report shows the cost of each day's usage, so it is clear where the budget is going.

This post was auto-generated by a LLM.

TrialGrid Version 89 - Broader Veeva Support and Improved AI Agents

Version 89 is a large release with significantly expanded support for Veeva studies. Our AI Agents have moved from answering one question at a time to authoring at scale and the machinery that runs Automated tests has become more resilient to interruption.

For more information see the release notes for Version 89.

Veeva: More of the Rule Language

The Veeva Test Case Advisor works by reading a rule and generating the test cases that exercise it, and its reach grows with the amount of Veeva's expression language it covers. Each release widens that coverage; this one widens it considerably.

The Advisor now understands the system attributes that real rules lean on: item-level values and units, the value an item held at the previous submit, a form's submit counter, change reason and status, and the Casebook, Study Country and Site attributes that describe the context a rule is running in. Site attributes were a particularly interesting problem, because a site is not known until the test is actually run - so a derivation that builds a value from the site name now embeds the real run-time value, looked up from Veeva and substituted automatically, while a rule that compares against a fixed site generates a scenario telling you which site the test must run on.

The Advisor also now handles rules that reach across linked forms - both counting links and reading data on the linked form - which it can do because the test language itself learned to link and unlink forms. That is the pattern for this release: the generator got smarter because the steps underneath it got richer.

A Fuller Vocabulary for Veeva Tests

A generated test case is built from the steps available to it, so the vocabulary of steps matters as much as the generator. Veeva tests can now add an Event Group to a subject, link and unlink forms, mark a form as intentionally left blank, and re-open a submitted form for edit - including re-opening automatically when a submitted form needs to be marked blank. These are everyday operations of running a Veeva study, and each one added is another part of the study your tests can speak to directly.

The editor around those steps has grown alongside them. Repeating event groups, forms and item groups each have a repeat input with validation of sequence numbers. The "has value" helpers suggest a field's codelist and dictionary values, in Rave as well as Veeva. Date fields keep what you type as you type it, so what you enter is what gets validated against the study's date format. And the editor now warns you when a step enters data into an item that is hidden by default, naming the item and value that would make it visible - so you can spot that at authoring time rather than at run time.

Agents That Author, and Agents That Tell You Where They Stop

Our Agents' work so far has been conversational: ask for an edit check, get an edit check. This release takes two steps beyond that.

The first is scale. The new bulk Edit Check Creator agent builds checks from specifications across a whole draft - it can fill in existing empty checks from the specification attached to them, or create new checks from custom specification objects. Every specification is classified by outcome - built with high confidence, built but needs review, needs a custom function, or failed - and those outcomes drive configurable action hooks, so the result of a bulk run feeds straight into your workflow. Results appear in the chat as they are built, so you can watch it work.

The second is judgment. Some specifications describe behavior that Rave's edit check steps do not express - regular expressions, extracting a component of a date, aggregating across records, conditions on form status. The Edit Check and Test Case agents now recognize those cases and tell you, so the boundary between what the agent can build and what needs a human is drawn explicitly, up front.

The same instinct shows up after the work is done. Having generated a check, the agent verifies that the logic it produced matches the specification it was given, and tells you if it does not. And when a specification names a custom function as handling the behavior, the agents build the check that calls it - noting if that function is not in the draft yet.

Runs That Look After Themselves

A test run depends on more than the test: the automated browser, the worker it runs on, and the environment around both. Those are the moving parts you have least control over, and the ones most exposed to interruption from outside. This release extends the runner's fault tolerance to cover them - if Chrome stops mid-run the runner restarts it and the run carries on, and a run whose browser is lost entirely is paused and picked up automatically on another worker. That is another class of interruption a run absorbs on its own, without anyone needing to watch for it.

Test running is smoother in other ways too. On the new Rave EDC, a step against a Folder that does not exist in the subject now names the Folder as the missing piece. Running a test set directly carries over the screenshot and continue-on-failure settings you chose last time in the wizard. Test Cases can now assert that an email is not received - matching on subject alone, subject with exact body, or subject with partial body.

In Summary

Veeva is where most of this release went: the Test Case Advisor now reads the bulk of the rule language you write day to day, and the test steps underneath it reach more of the operations a Veeva study performs. The test runner tolerates more of what goes on around it. And our Agents are being built to be clear about the boundaries of their work, because an agent that is open about where its work ends is one you can trust with the work it does.

This post was auto-generated by a LLM.