ICPSR's Secure Data Environment (SDE)

ICPSR has designed, built, and deployed what we call the Secure Data-processing Environment (SDE) over the past twelve months.  This is a tightly managed, highly controlled environment in which many members of the ICPSR staff perform their day-to-day data management (data processing) work.

The main business requirement behind the SDE is that it should be difficult, if not impossible, for content to leak out without a member of the staff taking an explicit action, such as running a program which formally releases content on the web site and commits it to archival storage.  For example, it should not be possible for someone to upload a data file into a web form, or to attach it to a piece of email.

The design called for many changes to ICPSR's technology infrastructure.  We separated our storage into two pools - Private (accessed within the SDE) and Semi-Private (which is more accessible).  We separated our network into three main virtual LANs - Private, Semi-Private, and Public.  We also updated many, many software systems so that they would operate properly within the SDE.  And we also changed processes to conform to the new business requirements.  For example, if one process required a data processor to send an email containing a data file to someone else at ICPSR, we changed the process so that email was not required.

I'll post a series of articles over the next few weeks with more details about the SDE and its technology.  This will include posts about how we separated storage; how we segmented the network; how we used virtualization technology to solve certain problems; how we changed key software systems; and, how the SDE changed business processes at ICPSR, and how it continues to do so even today.