ICPSR launches Measures of Effective Teaching web site

Some of my colleagues, including ICPSR Director George Alter, gave a demo of one of our newest Web sites and collections at the American Educational Research Association 2013 annual meeting on Sunday.

Click the image to navigate to the live site

My team has built the video portal portion of the system.  The portal enables a researcher to play a list of videos that s/he has selected to view based on an analysis of the associated quantitative data and tagging data.  Access to the video and datasets is restricted and requires one to complete a data use agreement via ICPSR's web-based request system.

We're grateful for the support we've received from the Bill and Melinda Gates Foundation to make all of this possible.

ICPSR web site maintenance

We ran into a few problems last Wednesday during our system update.  We rolled back the changes, and are giving it another go this evening at 10pm EDT.  We're going to move traffic to our replica in the cloud during the maintenance so that we have more time for troubleshooting.

Our cloud replica has many of the features of our main site (search, download, analyze), but does not include features that transfer materials to ICPSR, such as our online Deposit Form.

ICPSR web maintenance

We're updating a few pieces of core technology on our web server this afternoon:  httpd, mod_perl, Perl, and a few others.  Normally we like to perform maintenance like this during off-hours, but we're doing it at 12:30pm EDT today so that we have "all hands on-deck" to troubleshoot and solve problems.

We've already performed this maintenance on our staging server, and that went smoothly.  Our expectation is that this maintenance will last 15-30 minutes.

ICPSR system outage - 5/31/2012

ICPSR's content delivery systems faulted at approximately 8:30pm EDT on Thursday, May 31, 2012.  The oncall engineer discovered that the production Oracle database server had become unresponsive, and this disabled most features of most of our web portals.

After arriving on-site she rebooted the database server, but by then the production web server had become hopelessly confused.  She then rebooted that system as well, and all systems were back in service a bit before 9:30pm EDT.

The ICPSR technology team is reviewing system logs and access records to see if any further corrective action is required.

Our apologies for the inconvenience this no doubt caused to many of you.

SSLv2 and ICPSR

According to our web server logs, there aren't many of you who have been using SSLv2 when navigating to a URL that begins with "https" on the ICPSR web site.  And that's good.  Because this is a protocol which has known flaws for protecting information.

The ICPSR technology team disabled SSLv2 on its production web server last week.  Based on the logs we keep, it looks like this change will affect few, if any, web site visitors.

ICPSR network maintenance

This morning's network maintenance at the University of Michigan is running longer than expected.  ICPSR staff rolled service from the production systems to our replica in Amazon's cloud at 4:55am EST today.  The U-M data networking team now estimates that the work will be completed by 10:30am EST.  Once ICPSR confirms that the maintenance is complete, we will roll service back to the production systems.

You can track the maintenance using the same link we are - http://status.its.umich.edu/outage.php?id=73819

Network maintenance - Sunday morning (EST) Feb 12, 2012

The University of Michigan central IT organization, ITS, will be upgrading the network gear that connects ICPSR's building to the campus data network.  The work is scheduled to start at 6am (EST) on the morning of Feb 12, 2012 and should take between one and two hours.

The ICPSR IT team will redirect traffic from the production system to our cloud replica during the maintenance period.  The replica runs in Amazon's cloud and features services such as search, analyze, and download, but purposefully does not enable features such as deposit.

Between the cut-over to the replica, the network maintenance, and the fallback to the production system, access will likely be a little rocky next Sunday morning.  Like a freeway during construction, if it is possible to take a detour around ICPSR's web site on Sunday morning, that's the safest route.  But if you find that you need to download some data or use the site, the replica will be available.

Job posting - again

We're posting a job description for a senior software developer for a third time.  If there is a recession in the IT business in SE Michigan, somebody forgot to tell our pool of potential applicants.

This position is very much like other software developer positions at ICPSR.

In practice there is a blend of business analysis, system design, software development, and second-line on-going support for the stuff you write.  Building stuff in java to run under tomcat is a must.  Experience with Oracle, Eclipse, and one or more frameworks is very useful.

The main support for this position is our Bill and Melinda Gates Foundation MET Extension project.  This is a two year grant to build systems that will delivery video content to researchers in the social sciences and education.  We've had a lot of success - like a 0% failure rate - at hiring people to work on project X, and then moving them over to project Y a few years down the road.  Project X is in this case is MET Extension; project Y is unknown.  But so was the MET Extension project 10 months ago...

If you have interest in the position and would like more info, please feel free to drop me a note.  And here's a short-lived, but direct, link to the job http://umjobs.org/job_detail/66372/software_developer_senior

Tech@ICPSR talks about the cloud @ LA2M

Tech@ICPSR will be giving a talk on cloud computing at the February 1, 2011 LA2M meeting.  We'll be talking about the cloud; kind of a high-level overview of what different folks say the cloud is, and some of the consumer- and business-oriented services and systems that live in it.

I'll add a link to the materials shortly after the talk, and, if LA2M adds the video of the talk to their archive, I'll add a link to that as well.

Using DuraCloud for Archiving and Preservation

I'll be joining Michele Kimpton, CEO of DuraSpace, on a webinar next Wednesday (November 2, 2011).  Our topic is DuraCloud, and how one can use this cloud-based service as part of one's digital preservation strategy.

I think sometimes people will view the cloud as an alternative to keeping and maintaining local copies, but at ICPSR we're using the cloud as an easy-to-manage storage location to supplement more conventional locations, such as local NAS storage and the University of Michigan's "Value Storage" service.

Here is a copy of the invite that went out via email:

DuraSpace

You are invited to attend the following event: Using DuraCloud for Archiving & Preservation Wednesday, November 2, 2011 1:00p.m. - 2:00p.m. Eastern Standard Time Presented By: Michele Kimpton, DuraSpace Chief Executive Officer & DuraCloud Project Director & Bryan Beecher, Director of Computing & Network Services,Interuniversity Consortium for Political and Social Research (ICPSR) Having a hard time keeping up with current preservation and archiving practices? Are you finding the task of archiving your content complicated, costly and confusing? Then you need to join us for a free webinar that details how DuraCloud can be part of your preservation and archiving solution. This webinar will discuss how to use DuraCloud as a component of your archiving and preservation strategy. An overview of the service will include what it is, how it works, and the benefits it has to offer. Additionally, Bryan Beecher, Director of Computing & Network Services at ICPSR, will present ICPSR's preservation and archivingstrategy. Bryan will share how DuraCloud and other methods have been implemented to meet ICPSR's preservation and archiving goals. If you are interested in attending please thoroughly complete the registration process (below) to receive your unique login url. Be sure to SAVE the return email you receive from Infinite Conferencing as it will include your unique login information.   To register for this event, use the following link: http://ash-cs7.conferenceservers.com/brands/INF/EP_WebInterpoint/maskpath.html?id=66c84ec486&l=en-US  Below is the call-in information for the event. Via Skype (Free, World): Dial +9900827047086940  Via Phone (Toll, US): Dial +1(201)793-9022 Enter Room Number: 7086940 The maximum capacity for this web seminar is 99 participants. The event will be recorded and slides will be available for viewing after the event at http://duraspace.org/web_seminars. Please contact Kristi Searle at ksearle@duraspace.org with any questions. **Please be aware of Infinite Conferencing System Requirements: -          Internet connection speed of 128 kbps or higher is recommended -          Microsoft Windows XP, Vista, Windows 7, or Server 2003 -          Internet Explorer 6.0 SP2, 7.0, 8.0 & 9.0, Firefox 3.0x/3.5, 4, 5 and Chrome 12 browsers -          Apple Mac with Intel CPU, Mac OS X 10.5/10.6, Safari 4.x, 5.x or Firefox 3.x, Java 1.5+ -          Linux, Unix, or Solaris with Mozilla 1.0+ -          Cookies and Scripting enabled in browser

ICPSR wins grant from the Bill and Melinda Gates Foundation

ICPSR and partners at the University of Michigan received a grant from the Bill and Melinda Gates Foundation recently.  You can find the official link here.

 The link says that the grant is to house and make available to qualified researchers the data collected by the Measures of Effective Teaching project, and than means that you'll soon see yet another ICPSR operated web portal.

In addition to building and operating the portal, the project also requires us to process, preserve, and deliver quantitative data related to the Measures of Effective Teaching (MET) project, which is right in ICPSR's wheelhouse, of course.  The really new element for us, though, is the collection of video and "artifacts" related to the video.

ICPSR will use its existing Restricted Contract System (RCS) to screen applicants who want to access the video collection.  If approved the applicant will be able to access a video streaming server to view the videos in the collection.  The applicant will also be able to access the quantitative data in our Virtual Data Enclave (VDE).

The video collection is large compared to our current holdings of survey and government data.  My sense is that our collection will pretty much double in size, approaching 20TB total.  That's very big for us, but not nearly as big as some collections of video.

All Things Confidential

Tech@ICPSR will be at the University of Michigan's Michigan Union this Thursday to participate in the 2011 biennial meeting of our Organization Representatives from across the world of higher-education.  We'll be batting lead-off for the All Things Confidential session at 9am EDT.

If you cannot attend the meeting in person, you can still attend virtually.

Happy Birthday! Happy Anniversary!

It is the season for birthdays and anniversaries.

ICPSR is turning 50 years old, and will be hosting many events.  We launch the year-long celebration at our biennial meeting of our Official Representatives this October.  We'll host a reception at the American Political Science Association in September.  And you can also find us in Las Vegas at Caesar's Palace during this year's American Sociological Association meeting.  And, of course, there is our nifty new ICPSR@50 web site.

And what happens in Vegas stays in Vegas.

My tenure as the IT leader at ICPSR will turn 10 years old this fall.  Since 2002 the size of the software development team, which spends most of its effort building content delivery systems for grants, contracts, and our membership business, expanded from two to eight.  The number of servers - real and virtual - has expanded from two to two dozen (real) and two dozen (virtual).  The amount of disk storage available has increased from less than 1TB to over 100TB (and even more if we count off-site archival storage locations where the storage is managed by someone else). ICPSR has moved into social networking, cloud computing, disk-based archival storage for preservation, and virtualized access for sensitive data.  It has been a very exciting time!

And, finally, the tech@ICPSR blog turns 200 (posts) with this entry.

Amazon outage bites ICPSR too

Amazon reported a problem with its US-EAST region yesterday evening (EDT time).  Their service health dashboard reported that instances (virtual machines) in that region were having problems connecting to the Internet.  That was definitely true of our stuff.

We first saw alerts for our (virtual) equipment in the US-EAST at 22:25 EDT.  At that time we lost connectivity to every single instance in the US-EAST, but could still reach a small number of instances we have in other regions.  This affected the cloud-based replica of our production web server, the Teaching With Data web portal, and our "social login" service.  This latter service runs on an Amazon US-EAST operated by a company called Janrain, and isn't part of the instances where ICPSR has direct control.

By 22:56 EDT all of our systems were again reachable from the Internet, and no further action on our part (restoring content, restarting the instance) was necessary.

I have not yet seen a post mortem from Amazon, but based on my time in the data networking biz, my guess is that someone (Amazon or one of its transit or peering partners) made a routing change which blackholed their US-EAST traffic.

All in all Amazon continues to do a very good job with their cloud infrastructure, but this is a reminder that one would need to replicate services across several regions if one was to build a service with a very high level of availability.

Google and Facebook join MyData as one of the ways to login to ICPSR

After an unexpected delay we finally rolled out two new ways to authenticate to the ICPSR web site.  In addition to using the original ICPSR MyData account, web site visitors may now use either their Facebook account or Google ID to login to the ICPSR web site.

We wanted to make this type of service available because it is clear that most of our web site visitors who download data use it just once, or only infrequently.  And so for every researcher or scholar that uses ICPSR on a regular basis, there are two casual users who just want to download a dataset as quickly and easily as possible.

For this latter audience MyData doesn't make sense.  Why create a new account with a new password for something that may never be used again?

Now, people who would like to use their Facebook account or Google ID to identify themselves may do so.  And, better still, if the person is already logged into either service, they do not have to login again to ICPSR.  (This is often called SSO for Single Sign-On.)

We also looked at supporting the OpenID service, but our experience was that OpenIDs were a little too confusing for people.  The idea that one's identity is a URL rather than an email address is less common, and it felt too foreign to most people.

We hope the new login options are useful, and make it even easier to use ICPSR.

ICPSR Machine Room maintenance

I don't see an announcement about this on the main web site, so I'll drop a short note here....

The University of Michigan Plant Operations department is performing some electrical work in our machine room starting at 8am this morning.  The work will last some unknown number of hours; my guess is 4-6.

We have just diverted traffic from our production web site over to the replica in the cloud.  Our most common services such as search, download, and on-line analysis will be available.  But services which require our web site to "remember" something -- such as data deposit, Summer Program registration -- are not enabled in the replica.

I'll post a follow-up once the work has completed, and will check back here periodically for comments.  If you see something that isn't working as you think it should, please comment here, and we'll take a look.

----

10:22am EDT : We're aware that study search is not working.  (Variable search and bibliography search are working.)  ICPSR recently updated the search to include the full-text of many types of documents (codebooks, survey instruments, user guides, etc), and the index location changed too.  This change was not reflected on the cloud replica, and the index there is now building.  Our apologies for the inconvenience this may cause.

ETA for conclusion of the electrical work is between 11am and noon EDT.

-----

11:18am EDT: The power work has been completed.  We've restored service to the main site.

ICPSR Machine Room Work - Redux

The machine room power work has been completed, and we've been bringing our web services back on-line.  The ICPSR and Research Connections web sites are back in full service now.

ICPSR Machine Room Work

Well, it has been an interesting morning so far....

If you have been trying to use our web site this morning, then you know it got off to a slightly rocky start.  It turns out that an automated database copy between the primary server at ICPSR and the replica in the cloud malfunctioned in a new, unusual, and uncommon way.  And, of course, it happened this morning.  So while static web pages loaded just fine, content pulled from the database was not working fine at first.  Sorry about that, folks.

We're still having some challenges with the index behind our Research Connections site.  Part of the team has been checking into that while the rest has been moving equipment in our machine room.  When the metaphorical smoke clears later this afternoon, we'll have a major new supply of power in the machine room, and that will allow us to stand-up a new UPS system later this month.  And that will: (1) give us more head room when we use electrical service, (2) allow us to connect up some new systems that do not have UPS, and (3) get rid of a dozen old UPS systems, some of which we own, and some of which are rentals.

It sounds like we should be able to start restoring systems before noon today.  I'll include a follow-up post once we're fully back in business.

ICPSR Power Outage

The University of Michigan is upgrading the electrical system in the ICPSR machine room, and the final part of the project takes place this Saturday (January 15, 2011).  The U-M electricians will need to "de-energize" the machine room at 9am EST, and their work might take up to eight hours to complete.

My team will be on-site earlier that morning to shut down all of our key systems, and to transfer ICPSR's public-facing services to our replica in Amazon's computing cloud.  People using ICPSR's web site may experience brief interruptions in service as we transition control from the machine room to the cloud.  This will take place between 8am and 9am EST.

Once the U-M electricians have completed their work, we will transfer services back to our primary site at ICPSR, and web site visitors may experience brief interruptions during that time too.

ICPSR web site will be unavailable briefly on Monday, September 27, 2010

We've scheduled some maintenance around noon (EDT) on Monday, September 27, 2010.  We normally like to perform this type of work during off-hours, but this particular task is likely to be short-lived (10-15 minutes) and is best performed when we have our full team of software developers available.

My apologies in advance for any inconvenience.