home View on GitHub

The YES3 Exporter

User Guide, Technical Guide, Change Log and more

Change Log

Version 1.0.7

March 12, 2024

  1. README correct to reflect the new documentation model.

Version 1.0.6

February 26, 2024

  1. Modified as required for REDCap v14 compatibility. Tested on v14.1.4 (php 8.1.2) and v13.7.31 (php 7.4.3).
  2. Framework changed from 9 to 12.
  3. New documentation model: All documentation has been moved to a separate repository for which a GitHub pages site is deployed off the main branch. The “document explorer” plugin has been replaced by a link, identified with a “book reader” icon, to the GitHub pages site.
  4. Permission model changed, to distinguish between ‘exporter’ and ‘designer’ roles. Export ‘design’ permission is granted only to project designers. Export permission is granted to only users having REDcap export permissions on all forms involved in the export specification, regardless of designer or superuser status.
  5. Export specifications will be selected or excluded based on (1) the user’s design and export permissions and (2) the results of a basic export validation check prior to being loaded into the export manager. Export validation checks include:
    • verifying that an export specification is complete enough to be executed
    • verifying that all forms, fields and events are still defined on the project
    • for longitudinal projects, verifying that all forms are still assigned to events.
  6. Results of the permission and validation checks, if any, are reported to the browser console. The user is alerted to the presence of the console log and instructed to press Ctrl+Shift+J to access the browser console. In a future release the validation check will be stored with the export specification.
  7. Once an export specification is selected and loaded, a message will be posted indicating the user’s ‘design’ and ‘export’ permissions for that specification. The ‘download’ and ‘export to file system’ action icons will be disabled if the user lacks the necessary permissions.

Version 1.0.2

August 20, 2022

  1. basename() used to sanitize download filename headers
  2. getSafePath() used for file writes

Version 1.0.1

August 14, 2022

  1. Framework changed from 5 to 9; v12.0.4 now the minimum REDCap version
  2. Plugin CSRF token handling modified to meet F9 requirement
  3. Minor UI tweaks
  4. Client-side export column count estimation improved

Version 1.0.0

August 12, 2022

  1. Updates to README based on v0.9.3 feature changes
  2. Minor changes to technical.md

Version 0.9.3

August 10, 2022

  1. Data distribution columns removed from dd if the dd is downloaded by itself (no data export).
  2. An “add all forms and events” option added to the BULK ADD form.
  3. Bug fix: Wayback had displayed the “save” button even when no backup selected.
  4. The number of records downloaded is reported after a data download. Requires cookies to be enabled.
  5. Export specification included in log records for “export files written” entries. Log browser updated to include a specification inspection.
  6. Export log browser scrolls to the end of the list when opened.
  7. DAG logic in record selection queries improved

Version 0.9.2

August 5, 2022

  1. Dialog buttons rearranged to match REDCap (right justified, Save then Cancel).
  2. Rapid entry row now labeled QUICK ADD.
  3. New BULK ADD row placed below QUICK ADD; loads bulk insertion form.
  4. Weekly activity emails sent only if activity detected since last weekly roundup.
  5. Links to GitHub documents added to README so they can be reached from the REDCap EM management page.
  6. Item Editor mode text improved (“Item(s) will be inserted above selected row.” etc)
  7. Default event prefix algorithm changed to produce [e1, e2, … ek] if unique string length not determined (pattern was “e” + event_id code ).

Version 0.9.1

July 30, 2022

  1. Bug Fix: The ‘remove freetext fields’ option also removed the record id field(!).
  2. Edits to the help popups in the editor, prefixes and logs plugins.
  3. New selection criterion syntax for selecting all nonblank values.
  4. Bug Fix: Changing just the event in the Export Item Editor would sometimes be blocked with a ‘No can do: this item is already in the specification’ message
  5. Bug Fix: The bulk insertion action icon could become enabled on the Settings tab.
  6. Bug Fix: Export name not displayed in the Export Log Inspector.
  7. Export spec SELECT options are now displayed in order of export name.

Version 0.9.0

July 5, 2022

  1. The internal keyword for “all events” and “all forms” changed from “all” to “_all_” to allow forms named “all” to be exported.
  2. NEW ITEM form redesigned for better responsiveness.
  3. services.php updated to address taint warning.
  4. Minor bugs and code cleanup

Version 0.8.2

June 30, 2022

  1. Format of form autocomplete label changed to include the raw form_name value as well as the form label, i.e. same pattern as field autocomplete. Embedded space twixt the two components guarantees that spacebar will return all entries.
  2. Bug fix: reg expressions for event prefix, fieldname and filename validations corrected to allow single char names (3+ chars had been required).
  3. Support for date_dmy and datetime_dmy formats added to the variable type mappings.
  4. Bug fix: NEW ITEM (‘rapid entry’) form was dysfunctional for repeating form layouts
  5. Bug fix: designers were not accorded ‘exporter’ status - and hence couldn’t use the exporter - if denied export rights to any individual forms.
  6. Technical guide (documents/technical.md) added
  7. Document viewer redesigned to support readme, tech guide and changelog
  8. NEW ITEM now defaults to ‘form’, since it is more likely to be used to add forms to the spec.

Version 0.8.1

June 18, 2022

  1. Major README edits and reorganization
  2. Export log browser date range selectors no longer updated based on observed range
  3. Bug fix: Log record export failed
  4. Export log browser now displays the row count
  5. Export log browser UI tweaked for better responsiveness
  6. More thorough auditing of event prefixes:
    • must start with an alpha (a-z)
    • must end with an alphanumeric (a-z0-9)
    • alphanumerics and underscores allowed in between
    • cannot match another prefix (no duplicates) - caught on save
  7. More thorough auditing of export names, both when export created and when name is edited:
    • must start with an alpha (a-z)
    • must end with an alphanumeric (a-z0-9)
    • alphanumerics, spaces, hyphens and underscores allowed in between
    • cannot match another profile name (no duplicates)
  8. Bug fix: the ‘repeating form’ layout option was offered to projects not having repeating forms
  9. Bug fix: the export item list was improperly sized for non-longitudinal projects
  10. New ‘rapid entry’ NEW ITEM form located at the bottom of the export items list

Version 0.8.0

June 4, 2022

  1. Designers now accorded permissions on all forms, regardless of form export and view permissions
  2. Major CSS cleanup
  3. Option to activate beta features removed
  4. Exporter Main UI overhauled for efficiency, clarity and to maximize the amount of information displayed:
    • UI transformed from a single form with an embedded table having input fields for each item row (form/field name, event) to a list that invokes a single item editor
    • Ordering convention of item event, item object adopted throughout UI
    • Form titles (Followup MMSE) displayed instead of form names (followup_mmse).
    • Field labels are displayed as well as field names
  5. Numerous UI tweaks based on user testing

Version 0.7.5

May 25, 2022

  1. Bug fix: The selection criterion value “list” syntax failed with no records selected.
  2. Bug fix: The selection criterion expression permitted a “=>” operator instead of “>=”.
  3. Added “<>” to the set of allowed operators for the selection criterion.
  4. The selection criterion filter will perform numeric comparisons if the operand is numeric.
  5. UI: ‘labels always sanitized’ note added to editor.
  6. UI: The download dialog will now display a note about user data downloads being disabled, where appropriate.
  7. UI: A detailed help popup is available from the form insertion panel.
  8. Bug fix: It was possible to create duplicate export names in the settings editor.
  9. When a backup is loaded by the wayback, a message is displayed admonishing the user to save it in order to accept the selection.
  10. Confirmation is now required if you attempt to select another export while there are unsaved changes.
  11. Bug fix: Invalid REDCap field names were not always handled properly after change.
  12. YES3 logo added to the event preefix and log plugins.
  13. Bug fix: Action icons embedded in help popups were assigned click handlers.

Version 0.7.4

May 23, 2022

  1. Cron manager ‘yes3_exporter_cron’ redesigned. Now scheduled/reported in system settings instead of project settings. More consistent exception handling for cron tasks.
  2. New housekeeping function ‘hk_generations’ to cull old export specification backups.

Version 0.7.3

May 18, 2022

  1. Bug fix: Action icons that appeared to be disabled would under certain circumstances respond to clicks.
  2. Bug fix: Opening an export spec for a repeating layout would crash if the user was denied view or export rights on any form in the project.
  3. Debug help: Any Javascript error generated on one of our dev or staging hosts will now produce an onscreen report, reset the curser and shut down any modal dialogs.
  4. Bug fix: the count of data values written was incorrect in the logs (double counted).
  5. New plugin for displaying module markdown files, motivated by Parsedown apparently not supporting GitHub-flavored TOC tags. Uses GitHub CSS for light and dark themes, and has a sidebar TOC for navigation. Sidebar vanishes at the Bootstrap ‘small’ breakpoint. A vague plan is afoot to move all YES3 module documentation to a Docusaurus site.

Version 0.7.2

May 14, 2022

  1. Yes3::ellipsis() replaced by Yes3::truncate in all metadata contexts (ellipses taking up too much space when maxlen is small)
  2. Label sanitation function now strips all html tags, instead of just the ‘dangerous’ tags.
  3. Bug fix: UTF8 characters not rendering properly in MS Excel and other apps, because of missing byte order mark (BOM)
  4. Bug fix: it was possible, by blanking already-saved fields, to save an item with form or field name blank
  5. Bug fix: bulk insertion was not inserting above the selected field. Fixed to act consistently with the ‘paste’ function, if a single field is selected when the bulk insert is activated.
  6. Modal ‘busy’ message broadcast with ‘wait’ cursor style, for time-consuming AJAX processes (loading spec, saving spec, exporting data)
  7. Export to host and download functions separated, with new icons.
  8. Deprecated: The ‘export target’ setting is deprecated and no longer exposed on the UI.
  9. New EM setting: if filesystem exports are enabled, user data downloads can be prevented. Data dictionary downloads are allowed.
  10. Tailored and detailed non-modal help popup for selection criterion value
  11. Bug fix: several issues with vertical and repeating export layouts addressed: (1) event selection was inoperable (hangover from original version that did not allow event selections for these layouts); (2) no data exported for repeat layouts; (3) column orders were not the same between data dictionary and exported data; (4) repeating layouts allowed fields from multiple forms
  12. Export layout now appears as text, Instead of a disabled radio control.
  13. The form insertion dialog behaves slightly differently. If you select an event when “all forms” is selected, the dropdown for “all forms” becomes those forms for which the event is configured. Any insertion will be limited to those forms.
  14. The editor for the repeated layout behaves as for vertical, except that steps are taken to ensure that fields can only be selected from a single repeating form. Once you select one field or form, these constraint are applied: (1) the form insert icon is disabled and (2) the autocomplete for individual fields will include fields from that form only.
  15. Hovering over a completed field input will display the REDCap field label.