Creating a new model

Before creating a model on Evidencio, please make sure that you're signed in on Evidencio.
Now we can start with a new model.
Click on the tab models and then on New Model. A new screen will load and you start with the first step of the model creator.

A screenshot of the page: General

This is the screen you always start at when creating new models or editting existing models.

Title:
Start your model with filling in the title. The title is the only required field necessary for saving a model.

Author:
After saving the model at least one time, the author of the model is the user that is signed in.

Status:
There are three possibilities to choose:

  • Draft
  • Private
  • Public

If the model is saved on Public, an Evidencio employee gets a review request for that model. If this was not the intention, the review request can be withdrawn by saving the model on either Draft or Private. Make sure that the model is working properly before it is saved on Public.

Sharing:
It is possible to share your model with connections on Evidencio, this way you can let connections (such as co-workers) help you with the model-editing or let a connection review your model before saving it to public. It is necessary to save your model at least one time before sharing is possible.

Description:
This field is meant to describe the use of the model. It will often be the first thing anyone sees when they open a model. This field can give some context regarding the model or give some instructions about how to use the model.

Language:
Please select the language you use for the creation of the model.
note: this does not necessarily has to be the same language you've selected for the interface of Evidencio. (i.e. you can add an English model while using the Dutch version of Evidencio).

Specialties:
Please select the specialty that is most related to the topic of your model since it is allowed to select only one specialty.

MeSH terms:
Start typing in the MeSH term field and select multiple terms that belong to the topic of your model.

If you filled in all the fields, make sure to save your model and then go on to the next screen.

A screenshot of the page:Study
In this tab the composition of the study population is specified. The provided study characteristics can be used in our model validation tools.

Population:
Note the amount of patients used to develop the model, if possible specify amount of females and males. This information will eventually get a nice graphical visualization.

Additional information:
This field can be used to give additional context to the study population. For instance if the data was collected prospectively, where and when it was collected.

Study characteristics. Categorical characteristics:
By clicking the Add button, a small pop-up will show in which it is possible to name the characteristic, add subsets/groups and note how many patients were in that group.Click to see an example.

Study characteristics. Continuous characteristics:
Similar to the categorical characteristics, a pop-up screen appears when clicking the Add button. Continuous characteristics are either the mean or the median. If the Mean is chosen, the standard deviation has to be filled in. The minimum and maximum values are optional, but provide additonal functionality for model validation. If the median is chosen, the 1st and 3rd quartiles are necessary, and also the minimum and maximum are optional.

If all the fields are filled, go on to the next tab References, but first save your model!

A screenshot of the page:References
In this tab, all relevant sources to the development and possible external validations can be added. Information added in this tab can be seen directly in the list of all models.

Research Authors
These fields Name, Email, and Institute are used to enter the details of the authors of the developed model. Most published papers select one author for corresponding purposes. It is possible to notify the authors that their model was made available as an easy to use online calculator. The authors that have a correct Email address filled in and have a tick in the notify box get an email once the model was accepted as a public model.

Online resources
It is recommended to add the URL of the original paper in this section, such as a link to Pubmed or a DOI. Make sure to add Tags for the provided resources. The tags influence the eventual stars a model gets. It is possible to add multiple tags to a single resource.

Related files
It is possible to add relevent files over here such as an institute logo, make sure to add tags once a file has been uploaded. Selecting some of these tags such as the institute logo, gives the option to add a URL to the file.

Once all relevant sources have been added, proceed to the next tab Model. Make sure to save the model first.

A screenshot of the page:Model
This tab is the most important tab, since all the modeldetails such as the coefficients and intercept or the equation to get to an eventual outcome will be entered in this section.

Model Type
There are six model types to choose from. Select the model type that fits the model you want to add on Evidencio. This page provides a general description of the basic knowledge needed to add a model. To see a more detailed explanation on how to fill in the each of these model types, click on the model below. Results of all models are calculated differently.

The logistic regression and survival regression models can have standard coefficients or odds ratios as input type. The cox regression model alows hazard ratios. Odds and Hazard ratios are often described in published articles.

The linear, logistic regression, and survival regression models need use intercepts, where the cox regression model uses a baseline hazard. When selecting one of these model types, make sure that you have an intercept or baseline hazard. Without it, the model will not work.

The C-statistic is also known as the concordance index or the area under the receiving operator characteristic curve. This statistic shows how well the model performed on a validation.

If a model contains single coefficients, odds ratios, or hazard ratios, a choice can be made to use a multiplier or single values for the categorical variables. Note: Confidence intervals are only available with multiplier input.

Variables
Click the add button to add a new variable. Give the variable an easy to understand name and describe the variable in the description. The description is optional, but make sure that the variable cannot be misinterpreted when using the model. Select whether this variable is a categorical or a continuous variable and add the characteristics of the variable. Models might contain conversions in their described variables such as cubic splines. After adding the variable, it is possible to make conversions of that variable in the eventual calculation. For example: a model paramater “Age” is described with a beta coefficient of 0.05 for ages 18-50 and a beta coefficient of 0.09 for ages 51-80. First, add a continuous variable with the name age, minimum of 18, maximum of 80, and a beta coefficient of 0.05. Then make a conditional conversion where the condition age >50 converts the beta coefficient to 0.09.

It is also possible to make multiple formula segments which can be used for conversions or conditional results. A combination of different factors can be added to the segment. For instance, if height and weight of a patient are variables. Then a combination of height 185 cm and weight 30 kg seems unrealistic. A formula segment can be made where an unacceptable range can be added for feasible faulty inputs.

After all modelparameters are set, make sure to save the model and then go to the last tab Result

A screenshot of the page:Result
The result section contains valuable information regarding the interpretation of the calculated outcome. It is also possible to show additional information based on the input and calculated outcome.

Result settings The Pre-Result text should contain a small line of what the outcome means. E.g. “the calculated risk is”. Then add a little bit of context in the Post-Result text. E.g. “for developing lung cancer in the next 5 years” or . Combining the pre- and post-result texts, the eventual result looks like this: “The calculated risk is 12% for developing lung cancer in the next 5 years”. The result 12% in this example relies on the Result precision selected in the settings. If a decimal was added, the result might show 11.7%, for example.

The Result range allows the calculation of an outcome if one of the variables is unknown. The range will be calculated using two scenarios, using the minimal and the maximal input values of the unknown variable. For example, if the age of a patient is unknown and age is a continuous variable between 18 and 80, the range will be shown with a scenario in which the patient is 18 and a scenario in which the patient is 80. Note: Result range calculations may be inaccurate for nonlinear or complex formulas.

It is possible to show the complement of the result. For example, if a cox regression model predicts the risk of death over 5 years, the complement is the probability to not-die, thus showing the probability to survive.

The result interpretation field is one of the most important fields in the model. State what the clinical implications are of the calculated risk. Make sure that advises regarding the interpretation of the model are substantiated from available literature.

Outcome stratification
Based on the calculated risk, the outcome stratification can show additional information. Clicking the add button opens a pop-up in which it is possible to enter the additional information and between which interval. For example, some models give an indication whether a treatment might be beneficial. For example, if the predicted risk of lymph node metastases in prostate cancer patients is higher than 5%, a pelvic lymph node dissection is recommended. In the outcome stratification the range 5%-100% can be selected that shows a recommendation of the dissection if the predicted risk lies within this range.

Conditional information
Clicking the add button next to the conditional information opens a pop-up screen where additional information can be shown if certain conditions are met. For example, if a prediction of 50% or higher suggest the use of medication in a model, but that medication is useless in patients above 65 years old, then a combination of conditions can be selected.

If everything was added, then the model is ready to be used. Click on save model and then on show model. Test if the model runs like it should. If everything works properly, edit the model again, set it on public in the General tab and save it again to initiate the review proces. If the review was accepted, then the model can be used by anyone with an internet connection.

Click the video below to see how to create a model on Evidencio.