The beta dictionaries are used to be able to initiate creation of dictonaries in the harmonisation process of new variables in a very early stage. As a PI you can manage and create new dictionaries which can be uploaded and tested by involved cohorts in a pilot phase of the project. Dictionaries can be changed and updated at any time and the network will be unaffected. After testing and approving the final beta dictionary the variables will be included in the main dictionaries and released when in the 6 months cycle.

How it works

You need to have a few prerequisites

Working with ds-beta-dictionaries

You need to ‘clone’ the repository on your own workingstation to be able to create new dictionaries. This means you get a copy of the directory on your own system. You can do this by executing the following command:

git clone https://github.com/lifecycle-project/ds-beta-dictionaries

Now you have your own copy to work in.

Create dictionaries

Within the dictionaries-directory of the repository you need to create a dictionary for your study. This will result in a structure like:

  • ds-beta-dictionaries
    • dictionaries
      • yourstudy
        • dict_#flavor$
        • dict_#different flavor$

The flavors can be:

  • non_rep
  • weekly_rep (0-)
  • monthly_rep (0-219)
  • trimester (1-3)
  • yearly_rep (0-17)

Then you need to create a dictionary based upon the variables you want to harmonise. To illustrate how you do this you can use the example as a reference.

When you finished writing the dictionary you need to commit and push them to the repository on Github. Please check the git-workflow document to get the dictionaries exposed.

Upload dictionaries and data

We now support multiple backend with the beta-dictionaries.

  • Armadillo
  • Opal

Upload to Armadillo

When a first version of a beta dictionary is available, you can upload it to the Armadillo with the following code:

login_data <- data.frame(
  server = "https://armadillo.test.molgenis.org", 
  storage = "https://armadillo-minio.test.molgenis.org", 
  driver = "ArmadilloDriver"
)

du.login(login_data)
#>   Login to: "https://armadillo.test.molgenis.org"
#> [1] "We're opening a browser so you can log in with code 4G79WW"
#>   Logged on to: "https://armadillo.test.molgenis.org"
  
du.upload.beta(
  dict_name = "example-dictionary", 
  data_input_path = "/Users/sido/VisualStudioCodeProjects/ds-beta-dictionaries/data/example-dictionary/beta_dict_generated_data.csv",
  data_version = "1_0")
#> Error in du.upload.beta(dict_name = "example-dictionary", data_input_path = "/Users/sido/VisualStudioCodeProjects/ds-beta-dictionaries/data/example-dictionary/beta_dict_generated_data.csv", : unused argument (data_version = "1_0")

You will be able to check the data in the Armadillo immediately.

Upload to Opal

When a first version of a beta dictionary is available, you can upload it to Opal with the following code:

login_data <- data.frame(
  server = "https://opal.edge.molgenis.org", 
  password = "ouf0uPh6", 
  driver = "OpalDriver"
)

du.login(login_data)
#>   Login to: "https://opal.edge.molgenis.org"
#>   Logged on to: "https://opal.edge.molgenis.org"
  
du.upload.beta(
  dict_name = "example-dictionary", 
  data_input_path = "/Users/sido/VisualStudioCodeProjects/ds-beta-dictionaries/data/example-dictionary/beta_dict_generated_data.csv",
  data_version = "1_0")
#> Error in du.upload.beta(dict_name = "example-dictionary", data_input_path = "/Users/sido/VisualStudioCodeProjects/ds-beta-dictionaries/data/example-dictionary/beta_dict_generated_data.csv", : unused argument (data_version = "1_0")

After this you need to manually upload the data just like you would in a released data upload.

Be advised: the data is now placed into directories on the target system, in this case a “beta” directory