Web Policy  |  Intranet  |  Contact SCHEV    
Blogs
SCHEV Research Data Blog

The official blog of @SCHEVResearch at the State Council of Higher Education for Virginia. Discussions about our work, national higher education data policy, and highlights about the data we publish.

 

Recent Posts Blog Archives Subscribe Feed All Blogs

Using the Scorecard API

by Tod Massa 15. September 2015 12:01

I was going to do this post elsewhere, but Wordpress does not allow iFrames. So, we'll put it here as a professional sharing. With the release of the College Scorecard, USED and the Administration did two things of particular note: provided all the summary data behind the Scorecard for download and made it available through an API (Application Programming Interface). In the past I would have downloaded the data and loaded it onto my own database server, but I have tired of doing that. My frustration with changes in IPEDS structures over the years and the need to also go ahead and write stored procedures to pull the data safely (practicing good site design) leads to me question the value of such labor since this is really one of those "nice-to-do-things" (NTDT).

THIS ASSUMES YOU HAVE A BUSINESS INTELLIGENCE TOOL OR OTHER APPLICATION TO USE AND DISPLAY JSON DATA! If you don't,stop reading, unless you are really curious.

Here are the steps to use the API:

1) Go here: https://collegescorecard.ed.gov/data/ 

2) Read the page. And then this page - https://collegescorecard.ed.gov/data/documentation/

3) Go register to use the API and get your key. (The College Choice API is a GET API that lives at http://api.data.gov/ed/collegescorecard). There are five fields to complete, and then an API key will be mailed to you. It is simply a text string to include in each data request.

4) Download the data element dictionary.

5) Download and READ the Documentation Report. You have to do this know the years of the data available - which are key to specifying what fields you wish to pull as the year is part of the name specification.

6) Study the examples on the page.

For example:

https://api.data.gov/ed/collegescorecard/v1/schools?fields=school.name,id,2013.student.demographics.race_ethnicity.white,2013.student.demographics.race_ethnicity.black,2013.student.demographics.race_ethnicity.hispanic,2013.student.demographics.race_ethnicity.asian,2013.student.demographics.race_ethnicity.aian,2013.student.demographics.race_ethnicity.nhpi,2013.student.demographics.race_ethnicity.two_or_more,2013.student.demographics.race_ethnicity.non_resident_alien,2013.student.demographics.race_ethnicity.unknown,2013.student.demographics.race_ethnicity.white_non_hispanic,2013.student.demographics.race_ethnicity.black_non_hispanic,2013.student.demographics.race_ethnicity.asian_pacific_islander&sort=2013.completion.rate_suppressed.overall:desc
 

This sample pulls the completions by race/ethnicity for all schools in the database. It will not execute as it is because it needs an API code. Also, I don't every school, so I will insert a filter to the beginning if the statement in front of the &fields. (I damaged the api_key since it is registered to me, so this will not run.) The filter restricts the response to Virginia public four-year institutions.

https://api.data.gov/ed/collegescorecard/v1/schools?api_key=jpR7dT1gj22FM4mPrx8678uE9d7PGP3Oc2r5dK8w
&school.state=VA&school.ownership=1&fields=school.name,id,2013.student.demographics.race_ethnicity.white,2013.student.demographics.race_ethnicity.black,2013.student.demographics.race_ethnicity.hispanic,2013.student.demographics.race_ethnicity.asian,2013.student.demographics.race_ethnicity.aian,2013.student.demographics.race_ethnicity.nhpi,2013.student.demographics.race_ethnicity.two_or_more,2013.student.demographics.race_ethnicity.non_resident_alien,2013.student.demographics.race_ethnicity.unknown,2013.student.demographics.race_ethnicity.white_non_hispanic,2013.student.demographics.race_ethnicity.black_non_hispanic,2013.student.demographics.race_ethnicity.asian_pacific_islander&sort=2013.completion.rate_suppressed.overall:desc

When properly executed, the server returns the datastream below:

{"metadata":{"total":46,"page":0,"per_page":20},"results":[{"id":234076,"2013.student.demographics.race_ethnicity.two_or_more":0.0429,"2013.student.demographics.race_ethnicity.aian":0.0013,"2013.student.demographics.race_ethnicity.black":0.063,"2013.student.demographics.race_ethnicity.white":0.607,"2013.student.demographics.race_ethnicity.asian":0.1224,"2013.student.demographics.race_ethnicity.hispanic":0.0579,"2013.student.demographics.race_ethnicity.unknown":0.0486,"2013.student.demographics.race_ethnicity.non_resident_alien":0.0565,"school.name":"University of Virginia-Main Campus","2013.student.demographics.race_ethnicity.nhpi":0.0003},{"id":231624,"2013.student.demographics.race_ethnicity.two_or_more":0.0472,"2013.student.demographics.race_ethnicity.aian":0.0019,"2013.student.demographics.race_ethnicity.black":0.0714,"2013.student.demographics.race_ethnicity.white":0.595,"2013.student.demographics.race_ethnicity.asian":0.0682,"2013.student.demographics.race_ethnicity.hispanic":0.0914,"2013.student.demographics.race_ethnicity.unknown":0.0823,"2013.student.demographics.race_ethnicity.non_resident_alien":0.0419,"school.name":"College of William and Mary","2013.student.demographics.race_ethnicity.nhpi":0.0008},{"id":233921,"2013.student.demographics.race_ethnicity.two_or_more":0.0403,"2013.student.demographics.race_ethnicity.aian":0.0016,"2013.student.demographics.race_ethnicity.black":0.0349,"2013.student.demographics.race_ethnicity.white":0.7182,"2013.student.demographics.race_ethnicity.asian":0.0848,"2013.student.demographics.race_ethnicity.hispanic":0.0512,"2013.student.demographics.race_ethnicity.unknown":0.0295,"2013.student.demographics.race_ethnicity.non_resident_alien":0.0382,"school.name":"Virginia Polytechnic Institute and State University","2013.student.demographics.race_ethnicity.nhpi":0.0013},{"id":232423,"2013.student.demographics.race_ethnicity.two_or_more":0.0323,"2013.student.demographics.race_ethnicity.aian":0.0015,"2013.student.demographics.race_ethnicity.black":0.0399,"2013.student.demographics.race_ethnicity.white":0.788,"2013.student.demographics.race_ethnicity.asian":0.0414,"2013.student.demographics.race_ethnicity.hispanic":0.0472,"2013.student.demographics.race_ethnicity.unknown":0.0282,"2013.student.demographics.race_ethnicity.non_resident_alien":0.0195,"school.name":"James Madison University","2013.student.demographics.race_ethnicity.nhpi":0.0018},{"id":234359,"2013.student.demographics.race_ethnicity.two_or_more":0.0183,"2013.student.demographics.race_ethnicity.aian":0.0,"2013.student.demographics.race_ethnicity.black":0.2805,"2013.student.demographics.race_ethnicity.white":0.6402,"2013.student.demographics.race_ethnicity.asian":0.0305,"2013.student.demographics.race_ethnicity.hispanic":0.0244,"2013.student.demographics.race_ethnicity.unknown":0.0,"2013.student.demographics.race_ethnicity.non_resident_alien":0.0,"school.name":"Woodrow Wilson Rehabilitation Center","2013.student.demographics.race_ethnicity.nhpi":0.0061},{"id":446206,"2013.student.demographics.race_ethnicity.two_or_more":0.0303,"2013.student.demographics.race_ethnicity.aian":0.0,"2013.student.demographics.race_ethnicity.black":0.303,"2013.student.demographics.race_ethnicity.white":0.5455,"2013.student.demographics.race_ethnicity.asian":0.0,"2013.student.demographics.race_ethnicity.hispanic":0.1212,"2013.student.demographics.race_ethnicity.unknown":0.0,"2013.student.demographics.race_ethnicity.non_resident_alien":0.0,"school.name":"Culpeper Cosmetology Training Center","2013.student.demographics.race_ethnicity.nhpi":0.0},{"id":232681,"2013.student.demographics.race_ethnicity.two_or_more":0.0418,"2013.student.demographics.race_ethnicity.aian":0.0016,"2013.student.demographics.race_ethnicity.black":0.0597,"2013.student.demographics.race_ethnicity.white":0.6556,"2013.student.demographics.race_ethnicity.asian":0.0432,"2013.student.demographics.race_ethnicity.hispanic":0.0693,"2013.student.demographics.race_ethnicity.unknown":0.1225,"2013.student.demographics.race_ethnicity.non_resident_alien":0.0063,"school.name":"University of Mary Washington","2013.student.demographics.race_ethnicity.nhpi":0.0},{"id":234085,"2013.student.demographics.race_ethnicity.two_or_more":0.0155,"2013.student.demographics.race_ethnicity.aian":0.0018,"2013.student.demographics.race_ethnicity.black":0.043,"2013.student.demographics.race_ethnicity.white":0.8316,"2013.student.demographics.race_ethnicity.asian":0.0543,"2013.student.demographics.race_ethnicity.hispanic":0.0436,"2013.student.demographics.race_ethnicity.unknown":0.0,"2013.student.demographics.race_ethnicity.non_resident_alien":0.003,"school.name":"Virginia Military Institute","2013.student.demographics.race_ethnicity.nhpi":0.0072},{"id":234225,"2013.student.demographics.race_ethnicity.two_or_more":0.0,"2013.student.demographics.race_ethnicity.aian":0.0,"2013.student.demographics.race_ethnicity.black":0.037,"2013.student.demographics.race_ethnicity.white":0.963,"2013.student.demographics.race_ethnicity.asian":0.0,"2013.student.demographics.race_ethnicity.hispanic":0.0,"2013.student.demographics.race_ethnicity.unknown":0.0,"2013.student.demographics.race_ethnicity.non_resident_alien":0.0,"school.name":"Washington County Adult Skill Center","2013.student.demographics.race_ethnicity.nhpi":0.0},{"id":231712,"2013.student.demographics.race_ethnicity.two_or_more":0.0478,"2013.student.demographics.race_ethnicity.aian":0.0026,"2013.student.demographics.race_ethnicity.black":0.0793,"2013.student.demographics.race_ethnicity.white":0.7543,"2013.student.demographics.race_ethnicity.asian":0.0236,"2013.student.demographics.race_ethnicity.hispanic":0.0504,"2013.student.demographics.race_ethnicity.unknown":0.0378,"2013.student.demographics.race_ethnicity.non_resident_alien":0.0024,"school.name":"Christopher Newport University","2013.student.demographics.race_ethnicity.nhpi":0.002},{"id":232186,"2013.student.demographics.race_ethnicity.two_or_more":0.0461,"2013.student.demographics.race_ethnicity.aian":0.0016,"2013.student.demographics.race_ethnicity.black":0.0962,"2013.student.demographics.race_ethnicity.white":0.4576,"2013.student.demographics.race_ethnicity.asian":0.174,"2013.student.demographics.race_ethnicity.hispanic":0.1167,"2013.student.demographics.race_ethnicity.unknown":0.0613,"2013.student.demographics.race_ethnicity.non_resident_alien":0.0417,"school.name":"George Mason University","2013.student.demographics.race_ethnicity.nhpi":0.0048},{"id":232566,"2013.student.demographics.race_ethnicity.two_or_more":0.0351,"2013.student.demographics.race_ethnicity.aian":0.0028,"2013.student.demographics.race_ethnicity.black":0.0778,"2013.student.demographics.race_ethnicity.white":0.7822,"2013.student.demographics.race_ethnicity.asian":0.011,"2013.student.demographics.race_ethnicity.hispanic":0.0468,"2013.student.demographics.race_ethnicity.unknown":0.0351,"2013.student.demographics.race_ethnicity.non_resident_alien":0.0083,"school.name":"Longwood University","2013.student.demographics.race_ethnicity.nhpi":0.0009},{"id":233277,"2013.student.demographics.race_ethnicity.two_or_more":0.0412,"2013.student.demographics.race_ethnicity.aian":0.003,"2013.student.demographics.race_ethnicity.black":0.0969,"2013.student.demographics.race_ethnicity.white":0.7786,"2013.student.demographics.race_ethnicity.asian":0.0144,"2013.student.demographics.race_ethnicity.hispanic":0.0459,"2013.student.demographics.race_ethnicity.unknown":0.0122,"2013.student.demographics.race_ethnicity.non_resident_alien":0.0053,"school.name":"Radford University","2013.student.demographics.race_ethnicity.nhpi":0.0025},{"id":234030,"2013.student.demographics.race_ethnicity.two_or_more":0.0463,"2013.student.demographics.race_ethnicity.aian":0.0026,"2013.student.demographics.race_ethnicity.black":0.1842,"2013.student.demographics.race_ethnicity.white":0.5162,"2013.student.demographics.race_ethnicity.asian":0.1213,"2013.student.demographics.race_ethnicity.hispanic":0.0754,"2013.student.demographics.race_ethnicity.unknown":0.022,"2013.student.demographics.race_ethnicity.non_resident_alien":0.0293,"school.name":"Virginia Commonwealth University","2013.student.demographics.race_ethnicity.nhpi":0.0029},{"id":232982,"2013.student.demographics.race_ethnicity.two_or_more":0.0551,"2013.student.demographics.race_ethnicity.aian":0.0043,"2013.student.demographics.race_ethnicity.black":0.2588,"2013.student.demographics.race_ethnicity.white":0.5207,"2013.student.demographics.race_ethnicity.asian":0.0443,"2013.student.demographics.race_ethnicity.hispanic":0.0662,"2013.student.demographics.race_ethnicity.unknown":0.0335,"2013.student.demographics.race_ethnicity.non_resident_alien":0.0123,"school.name":"Old Dominion University","2013.student.demographics.race_ethnicity.nhpi":0.0048},{"id":233897,"2013.student.demographics.race_ethnicity.two_or_more":0.0044,"2013.student.demographics.race_ethnicity.aian":0.0032,"2013.student.demographics.race_ethnicity.black":0.1028,"2013.student.demographics.race_ethnicity.white":0.8006,"2013.student.demographics.race_ethnicity.asian":0.0082,"2013.student.demographics.race_ethnicity.hispanic":0.0196,"2013.student.demographics.race_ethnicity.unknown":0.0606,"2013.student.demographics.race_ethnicity.non_resident_alien":0.0006,"school.name":"The University of Virginia's College at Wise","2013.student.demographics.race_ethnicity.nhpi":0.0},{"id":234155,"2013.student.demographics.race_ethnicity.two_or_more":0.0,"2013.student.demographics.race_ethnicity.aian":0.0028,"2013.student.demographics.race_ethnicity.black":0.8561,"2013.student.demographics.race_ethnicity.white":0.0231,"2013.student.demographics.race_ethnicity.asian":0.0026,"2013.student.demographics.race_ethnicity.hispanic":0.0185,"2013.student.demographics.race_ethnicity.unknown":0.096,"2013.student.demographics.race_ethnicity.non_resident_alien":0.001,"school.name":"Virginia State University","2013.student.demographics.race_ethnicity.nhpi":0.0},{"id":234377,"2013.student.demographics.race_ethnicity.two_or_more":0.013,"2013.student.demographics.race_ethnicity.aian":0.0034,"2013.student.demographics.race_ethnicity.black":0.0954,"2013.student.demographics.race_ethnicity.white":0.8608,"2013.student.demographics.race_ethnicity.asian":0.0039,"2013.student.demographics.race_ethnicity.hispanic":0.0217,"2013.student.demographics.race_ethnicity.unknown":0.0019,"2013.student.demographics.race_ethnicity.non_resident_alien":0.0,"school.name":"Wytheville Community College","2013.student.demographics.race_ethnicity.nhpi":0.0},{"id":232937,"2013.student.demographics.race_ethnicity.two_or_more":0.0291,"2013.student.demographics.race_ethnicity.aian":0.0022,"2013.student.demographics.race_ethnicity.black":0.8381,"2013.student.demographics.race_ethnicity.white":0.0502,"2013.student.demographics.race_ethnicity.asian":0.0049,"2013.student.demographics.race_ethnicity.hispanic":0.0262,"2013.student.demographics.race_ethnicity.unknown":0.0432,"2013.student.demographics.race_ethnicity.non_resident_alien":0.0047,"school.name":"Norfolk State University","2013.student.demographics.race_ethnicity.nhpi":0.0015},{"id":231873,"2013.student.demographics.race_ethnicity.two_or_more":0.0191,"2013.student.demographics.race_ethnicity.aian":0.0076,"2013.student.demographics.race_ethnicity.black":0.0508,"2013.student.demographics.race_ethnicity.white":0.8793,"2013.student.demographics.race_ethnicity.asian":0.0013,"2013.student.demographics.race_ethnicity.hispanic":0.0216,"2013.student.demographics.race_ethnicity.unknown":0.0203,"2013.student.demographics.race_ethnicity.non_resident_alien":0.0,"school.name":"Dabney S Lancaster Community College","2013.student.demographics.race_ethnicity.nhpi":0.0}]}

Not easily usable, right?

This is why you need a tool.

In any event, i'm not interested in retrieving data I already have. It is the loan and repayment data that I find most interesting. So I have specified the three-year repayment rates of various subgroups. I then store the API request in a LogiAnalytics JSON datalayer, apply a "dataflattener" to it and then I can build a table just as if I was working in SQL.

 

 

Tags:

Categories:

blog comments powered by Disqus
Follow SCHEV Research on Twitter