← projects

Generic Survey Engine for Clinical Researches

A clinical research data collection platform for a healthcare group

A Generic Survey Engine for Clinical Research

A healthcare group was collecting clinical research data across different disease areas - hypertension, cardiology, endocrinology. Every new protocol meant building a new form, provisioning new tables, and writing new reports from scratch. I built a single engine where a research protocol is defined by data, not code.


The Problem

Clinical research platforms usually get stuck between two extremes:

The rigid approach - Separate tables, separate forms, separate schemas for every protocol. Safe, but slow. Launching a new study takes weeks: IT tickets get filed, columns get added, forms get designed and published.

The flexible approach - Generic form builders. Fast, but rarely up to clinical standards: weak typing, thin validation, painful reporting.

The client wanted the best of both: defining a new study should take an afternoon, but the data coming in should be as disciplined as on day one.

The platform decision had already been made: Microsoft Power Platform. That comes with a web resource constraint - the entire UI has to fit in a single HTML file. You can't use a module system, a build pipeline, or npm packages; you're working with plain vanilla JavaScript.


The Key Decision: Model Questions as Data, Not Schema

This was the pivotal choice. In the conventional approach, "patient's blood pressure" becomes a column. I made it a row.

erDiagram
    RESEARCH ||--o{ SECTION : contains
    SECTION ||--o{ QUESTION : contains
    QUESTION ||--o{ QUESTION_OPTION : offers
    QUESTION ||--o{ QUESTION : "related to"
    SESSION ||--o{ ANSWER : records
    ANSWER }o--|| QUESTION : answers
    ANSWER }o--o| QUESTION_OPTION : "selected value"

    RESEARCH {
        string name
        string status
    }
    QUESTION {
        string label
        int type
        bool required
        guid relatedQuestionId
    }
    ANSWER {
        string textValue
        decimal numericValue
        guid optionRef
    }

Because of this, adding a new research protocol no longer requires a code change. A coordinator defines sections, questions, and options from the admin app, and the engine reads them and renders the form.

After the first hypertension protocol shipped, the second study was defined in a single afternoon - without a developer in the loop.


Architecture

Three layers, each with a clear responsibility and loose coupling between them.

1. Data Layer (Dataverse)

Six custom tables: Research, Section, Question, QuestionOption, Session, Answer. Question types (single-select, multi-select, numeric, date, free text, Yes/No, Yes/No/Unknown) are modeled as an enum. There's no separate column per answer type - textValue, numericValue, and optionRef get populated depending on the question type.

2. The Engine (Web Resource)

One HTML file. Inside:

  • Renderer - picks the right UI widget for each question type
  • State manager - holds the active session's answers in memory and validates them
  • Persistence layer - writes to Dataverse via OData
  • Export - generates Excel output via SheetJS (desktop-only; mobile WebView can't handle downloads)

3. App Shell (Model-Driven Apps)

Two separate MDAs: one full-access admin app, one restricted doctor-facing app. The engine runs as an iframe inside both. Role-based visibility is enforced through Dataverse security roles - the engine derives what each user sees from their role, not from hardcoded logic.


Technical Deep Dives

Conditional Fields and Self-Referencing Lookups

If a physician marks a medication as "being taken," they should also fill in the dosage schedule. But if they're not, that field shouldn't appear - or show up in validation.

To handle this, I added a self-referencing lookup on the question table. The dosage question is linked to the medication it belongs to, and the engine follows that link when deciding what to render.

// In Dataverse, a lookup field has a different name in $select
// than in $expand (navigation property) - finding this trap
// cost me half a day.

const query = `?$select=name,type,required,_relatedQuestionId_value` +
              `&$expand=options($select=label,value)`;

// _relatedQuestionId_value ← this one in $select
// relatedQuestion         ← a different name in $expand

OData field naming in Dataverse isn't consistent. Lookup fields need the _fieldname_value format in $select but the navigation property name in $expand. Get it wrong and you don't get an error - you get an empty array, which means the problem can sit there unnoticed for days.

Yes / No / Unknown - A Three-State Boolean

Clinical data has a truth most systems ignore: "Unknown" is an answer. It isn't null. The patient was asked, they don't remember - that's information.

So I introduced a new question type: YESNOUNK. Three buttons, three distinct values (1, 0, 9). "Unknown" gets its own CSS class (selected-unknown) because the color semantics matter: yes/no is a decisive answer (navy blue), unknown is missing information (grey).

.answer-button.selected       { background: var(--brand-primary); color: white; }
.answer-button.selected-unknown { background: var(--neutral-500); color: white; }

The Falsy-Value Trap

Numeric fields were showing up empty in exports, and it turned out to be a classic JavaScript mistake:

// Wrong: 0 gets treated as "empty"
const exportValue = answer.numericValue || "-";

// Right: only null/undefined should count as empty
const exportValue = answer.numericValue ?? "-";

In clinical data, 0 is a meaningful value (e.g., non-smoker → 0 packs/day). If you use || here, you're losing data without ever knowing it.

The isSaving Guard and Failed Validation

The "Submit" button used an isSaving flag to prevent double-clicks. In the first version, the flag was set before validation. The result: whenever validation failed, the button stayed locked, and users reported a "frozen screen."

// Broken flow
isSaving = true;
if (!validate()) return;  // flag never cleared on this path
await save();

// Correct flow
if (!validate()) return;
try {
  isSaving = true;
  await save();
} finally {
  isSaving = false;
}

All it took was swapping two lines, but without that swap, users were convinced the screen had frozen every time validation failed.


Security Model

Three roles: Admin, Coordinator, Doctor. Different visibility, different permissions for each.

| Resource | Admin | Coordinator | Doctor | |----------------------|---------|-------------|---------------| | Research definition | Write | Write | Own only | | Question / Option | Write | Write | Read | | Session | All | All | Own only | | Answer | All | All | Own only | | Report / Export | Yes | Yes | No |

Dataverse security roles covered most of this matrix, but two subtleties showed up:

On the Session table, I had to grant doctors organization-level read - because they need to know other doctors' sessions exist (for duplicate checks), but not see inside them. The visibility restriction gets enforced in code via an ownerFilter, not in the schema.

The Append To privilege turned out to be another subtle point: to write a lookup via @odata.bind, the target table needs Append To = Organization. That doesn't make the table itself writable - it just says "this table can be referenced." Easy to miss in the docs.


Mobile Support

Doctors use tablets and phones in the field. The engine is responsive, but with one catch: the medication and dosage questions work best as a table view on desktop, which falls apart on narrow screens.

I fixed it with CSS alone. Below the breakpoint, table rows turn into cards via display: block, and each cell picks up its header label through ::before and a data-label attribute.

@media (max-width: 600px) {
  .drug-table, .drug-table tbody, .drug-table tr, .drug-table td {
    display: block;
  }
  .drug-table td::before {
    content: attr(data-label);
    font-weight: 600;
    display: block;
  }
}

The export function is hidden on mobile entirely - Power Apps mobile WebView doesn't support file downloads. Leaving the button there would mean users clicking it and watching nothing happen, so we hide it and show a toast instead: "Export is available on desktop."


Localization

The entire user-facing UI is in Turkish. But here's the subtler piece: Dataverse security role names are localized too. "System Administrator" becomes "Sistem Yöneticisi" in our environment. The engine's canViewAllSessions() check recognizes both.

const ADMIN_ROLE_NAMES = ["System Administrator", "Sistem Yöneticisi"];

It looks like a minor detail from the outside, but it keeps the engine from breaking silently when the environment switches locale, or later when someone spins up an English-language test tenant.


By the Numbers

6Custom tables1HTML file (entire engine)7Supported question typesDefinable research protocols

What I Learned

1. Low-code isn't no-code. Power Platform does speed up the business-rule layer. But state management, error handling, idempotency - the usual software concerns are still sitting there waiting for you. Drag and drop gets you to a starting point; everything that makes it a product comes after that.

2. The line between schema and data is a design decision. "Should this be a column or a row?" isn't an innocent question. Columns lock you to a shape; rows give you flexibility but cost you in performance and validation complexity. Rows were the right call for this project, but what mattered was knowing why I was choosing that way.

3. Finding gaps in enterprise documentation is part of the job. OData lookup naming, Append To semantics, the mobile WebView download restriction - none of these were clearly documented. You spend a real amount of time just figuring out what question to ask before you can look for an answer.

4. Physicians are the toughest users. They work fast, their patience is short, and they don't tolerate small friction. One locked button (like the isSaving bug) is enough to break their trust in the whole product. The invisible behavior of the UI needs as much care as the parts you can see.


What's Next