Musicboard Export
Building a data export tool for the popular Musicboard website
If you're a Musicboard user looking to export your data, you can use the Musicboard Export Tool tool. The following post provides technical details about the tool and how it works.
The popular Musicboard website has been down for a few days. The service promises that the website and mobile apps will once again be operational in a few days, but five days later that promise seems delayed at best.
After a little investigation, I discovered that there's a data scraper written in Python by Github user alexlawless12. This scraper relies on the website's UI to be up and running, so it is currently not functional. However, I have been able to use it as a starting point to build a data export tool that can extract data from the Musicboard API.
At this time, the app appears down due to an expired certificate on the API at https://api.musicboard.app/v2. The export tool (musicboardexport.bytecreator.dev) is a Next.js app that queries the Musicboard API from the server with the popular axios library. The rejectUnauthorized Axios configuration option allows the server to ignore the expired certificate. A key discovery is the query param at /v2/users?username={username}. The JSON from this endpoint contains the user GUID, which can then be used to query other endpoints that require the user ID. The export tool queries various endpoints to gather data about the user's profile, reviews, ratings, lists, and more. The resulting data is then compiled into the final CSV and JSON files that the user can download.
Appendix: API Endpoints
If you're interested in exploring the API, you can get started using PowerShell with the following command (note the -SkipCertificateCheck flag!):
Invoke-WebRequest -Uri "https://api.musicboard.app/v2" -SkipCertificateCheck -Outfile "response.json"The following API samples are just a few of the endpoints that can be queried. The API is extensive, and there are many endpoints that I haven't explored yet.
api.musicboard.app/v2/users
The users endpoint returns a paginated list of users. Each user object contains various details about the user, such as their username, name, biography, website, social media links, etc. This pagination appears to be configurable to pull larger batches. Another interesting tidbit is that there are a total of 763,340 users in the database.
{
"count": 763340,
"next": "http://api.musicboard.app/v2/users/?limit=24&offset=24",
"previous": null,
"results": [
{
"uid": "<PLACEHOLDER_GUID>",
"username": "<PLACEHOLDER_USERNAME>",
"name": "<PLACEHOLDER_NAME>",
"biography": "<PLACEHOLDER_BIOGRAPHY>",
"website": "<PLACEHOLDER_WEBSITE>",
"twitter": "<PLACEHOLDER_TWITTER>",
"profile_picture": "<PLACEHOLDER_PROFILE_PICTURE>",
"profile_picture_small": "<PLACEHOLDER_PROFILE_PICTURE_SMALL>",
"profile_picture_medium": "<PLACEHOLDER_PROFILE_PICTURE_MEDIUM>",
"profile_picture_large": "<PLACEHOLDER_PROFILE_PICTURE_LARGE>",
"is_pro": false,
"tier": null,
"is_verified": false,
"is_moderator": false,
"is_user_moderator": false,
"is_content_moderator": false,
"is_database_moderator": false,
"contributor_tier": null,
"background": null,
"profile_frame": null,
"profile_effect": null,
"color_theme": null,
"use_theme_on_content": true
},
]
}api.musicboard.app/v2/users?username={username}
This endpoint allows you to query a user by their username. The resulting JSON contains mostly unhelpful info, with the exception of the uid field, which is a GUID that can be used to query other endpoints that require the user ID.
{
"count": 1,
"next": null,
"previous": null,
"results": [
{
"uid": "<PLACEHOLDER_GUID>",
"username": "<PLACEHOLDER_USERNAME>",
"name": "<PLACEHOLDER_NAME>",
"biography": "<PLACEHOLDER_BIOGRAPHY>",
"website": "<PLACEHOLDER_WEBSITE>",
"twitter": "<PLACEHOLDER_TWITTER>",
"profile_picture": "<PLACEHOLDER_PROFILE_PICTURE>",
"profile_picture_small": "<PLACEHOLDER_PROFILE_PICTURE_SMALL>",
"profile_picture_medium": "<PLACEHOLDER_PROFILE_PICTURE_MEDIUM>",
"profile_picture_large": "<PLACEHOLDER_PROFILE_PICTURE_LARGE>",
"is_pro": false,
"tier": null,
"is_verified": false,
"is_moderator": false,
"is_user_moderator": false,
"is_content_moderator": false,
"is_database_moderator": false,
"contributor_tier": null,
"background": null,
"profile_frame": null,
"profile_effect": null,
"color_theme": null,
"use_theme_on_content": true
}
]
}api.musicboard.app/v2
Note that some of the following endpoints require authentication.
{
"activity": "http://api.musicboard.app/v2/activity/",
"patch_notes": "http://api.musicboard.app/v2/patch_notes/",
"users": "http://api.musicboard.app/v2/users/",
"most_active_users": "http://api.musicboard.app/v2/most_active_users/",
"follows": "http://api.musicboard.app/v2/follows/",
"blocks": "http://api.musicboard.app/v2/blocks/",
"wants": "http://api.musicboard.app/v2/wants/",
"favorite_albums": "http://api.musicboard.app/v2/favorite_albums/",
"my_wants": "http://api.musicboard.app/v2/my_wants/",
"favorite_genres": "http://api.musicboard.app/v2/favorite_genres/",
"datasets": "http://api.musicboard.app/v2/datasets/",
"tracks": "http://api.musicboard.app/v2/tracks/",
"albums": "http://api.musicboard.app/v2/albums/",
"artists": "http://api.musicboard.app/v2/artists/",
"dz_tracks": "http://api.musicboard.app/v2/dz_tracks/",
"dz_albums": "http://api.musicboard.app/v2/dz_albums/",
"dz_artists": "http://api.musicboard.app/v2/dz_artists/",
"genres": "http://api.musicboard.app/v2/genres/",
"styles": "http://api.musicboard.app/v2/styles/",
"ratings": "http://api.musicboard.app/v2/ratings/",
"my_ratings": "http://api.musicboard.app/v2/my_ratings/",
"reviews": "http://api.musicboard.app/v2/reviews/",
"most_liked_reviews": "http://api.musicboard.app/v2/most_liked_reviews/",
"lists": "http://api.musicboard.app/v2/lists/",
"most_liked_lists": "http://api.musicboard.app/v2/most_liked_lists/",
"list_connections": "http://api.musicboard.app/v2/list_connections/",
"comments": "http://api.musicboard.app/v2/comments/",
"likes": "http://api.musicboard.app/v2/likes/",
"my_likes": "http://api.musicboard.app/v2/my_likes/",
"reports": "http://api.musicboard.app/v2/reports/",
"drafts": "http://api.musicboard.app/v2/drafts/",
"stickers": "http://api.musicboard.app/v2/stickers/",
"threads": "http://api.musicboard.app/v2/threads/",
"poll_answers": "http://api.musicboard.app/v2/poll_answers/",
"featured_reviews": "http://api.musicboard.app/v2/featured_reviews/",
"featured_lists": "http://api.musicboard.app/v2/featured_lists/",
"featured_profiles": "http://api.musicboard.app/v2/featured_profiles/",
"discover": "http://api.musicboard.app/v2/discover/",
"promos": "http://api.musicboard.app/v2/promos/",
"push_notifications/device/apns": "http://api.musicboard.app/v2/push_notifications/device/apns/",
"push_notifications/device/gcm": "http://api.musicboard.app/v2/push_notifications/device/gcm/",
"subscriptions": "http://api.musicboard.app/v2/subscriptions/",
"products": "http://api.musicboard.app/v2/products/",
"ios_receipt": "http://api.musicboard.app/v2/ios_receipt/",
"android_receipt": "http://api.musicboard.app/v2/android_receipt/",
"pro_gifts": "http://api.musicboard.app/v2/pro_gifts/",
"backgrounds": "http://api.musicboard.app/v2/backgrounds/",
"articles": "http://api.musicboard.app/v2/articles/",
"suggested_album_changes": "http://api.musicboard.app/v2/suggested_album_changes/",
"suggested_track_changes": "http://api.musicboard.app/v2/suggested_track_changes/",
"suggested_artist_changes": "http://api.musicboard.app/v2/suggested_artist_changes/",
"suggested_albums": "http://api.musicboard.app/v2/suggested_albums/",
"suggested_tracks": "http://api.musicboard.app/v2/suggested_tracks/",
"suggested_artists": "http://api.musicboard.app/v2/suggested_artists/",
"suggested_artist_relations": "http://api.musicboard.app/v2/suggested_artist_relations/",
"suggested_artist_also_known_as": "http://api.musicboard.app/v2/suggested_artist_also_known_as/",
"suggested_artist_member_of": "http://api.musicboard.app/v2/suggested_artist_member_of/",
"suggested_artist_band_member": "http://api.musicboard.app/v2/suggested_artist_band_member/",
"suggested_track_composers": "http://api.musicboard.app/v2/suggested_track_composers/",
"suggested_track_writers": "http://api.musicboard.app/v2/suggested_track_writers/",
"suggested_track_staff": "http://api.musicboard.app/v2/suggested_track_staff/",
"messages": "http://api.musicboard.app/v2/messages/",
"groups": "http://api.musicboard.app/v2/groups/",
"group_settings": "http://api.musicboard.app/v2/group_settings/",
"posts": "http://api.musicboard.app/v2/posts/",
"transcriptions": "http://api.musicboard.app/v2/transcriptions/"
}