How does updating Retrospect work?

There has been a lot of confusion on how Retrospect update work. So this KB article is an attempt to explain the main ideas and principles behind the system. And the most important things to remember are:

1. Kodi caches stuff in the addons/packages/ folder and will always (yes always) use a file from that location.
2. Retrospect has 2 add-on types: the main add-on and the channel update add-ons. The main add-on will always include all update channels add-ons.
3. Downloads are throttled and take about 6 hours to reach all users.

How Kodi works

Kodi install add-ons from either zip files, or from repositories. Retrospect has its own repository that should be use to install Retrospect.  These repositories contain two very import files:

• addons.xml
• addons.xml.md5

Every 10 minutes Kodi checks the addons.xml.md5 and compares that found hash value in the file with the last known hash value Kodi has for that repository. If the hash differs, Kodi will download the actual addons.xml file. This file contains a list off all available add-ons within a specific repository.

It compares all add-ons it he addons.xml with the ones installed. If there is a newer version available for an add-on, Kodi will (if configured correctly):

1. Check if the addons/packages/ folder already contains that specific version (it might have been download before and have been un-installed)
   • If it is not available, it will automatically download the new add-on version and store the zip file containing the new version in the addons/packages/ folder of the main Kodi master profile
   • If it is available, it won’t download it and use the already present zip file.
2. Kodi will then install the new add-on version from the zip in the addons/packages/ folder into the actual addons/ folder

It is important to keep in mind that Kodi stores download packages in the addons/packages/ folder and will always check if a file exists there. Even if you were to use the “install from zip” option, it will still go to the addons/packages/ folder and take a equally named zip file from that folder!

How Retrospect uses this

Retrospect has two different types of add-ons:

1. The main Retrospect packages: plugin.video.retrospect-x.x.x.x.zip: These package contain the most recent version of the Retrospect Framework (basically all code that is needed to run Retrospect) and all available channels. These channels are automatically installed as add-ons within Kodi on the first run of Retrospect.
2. Channel update packages: plugin.video.retrospect.channel.<channelId>-x.x.x.x.zip that contain only channel updates.

Please not the difference in version numbering: the main Retrospect add-on always comes with a 3 digit version (E.g. version 4.1.3) and the channel updates always come with a 4 digit version (E.g. version 4.1.3.1). The main rule of thumb here is that the first 3 digits of the channel add-ons should be the same as the 3 digits of the main Retrospect add-on to be compatible. Retrospect will check for that when it runs and will ignore all channel add-ons that do not match this.

It is also important to note that the available channels that are being deployed will NOT be enabled by Kodi 17 and higher due to the add-on restrictions of third party add-ons. Retrospect will use the channels but that is because it does not care it is not an enabled add-on. However disabled add-on are not auto-updated! So don’t forget to enable all Retrospect add-ons within Kodi by going to:

• Settings
• Add-ons
• My Add-ons
• All
• And then enable all Retrospect related add-ons

Ok, so far it seems all rather standard (main add-on vs channel add-ons) and to be honest it is. However, what happens if a channel update is released?

Different Retrospect Add-ons in the Retrospect Repository

Channel Updates

As mentioned before, with the first run of Retrospect all channels are installed as Kodi add-ons (often grouped by language or purpose). This allows them to be updated independently of the Retrospect itself. Especially on the bandwidth this saves a lot! So whenever there is a channel update two main things happen:

1. A new version of the channel add-on is published in the repository. This version will appear in the repository as Retrospect <channel add-on name> (Update).
2. The main add-on is updated with this new channel add-on but we don’t update the version, as that would trigger a full update for all users.

The main reason for this last step is that it saves new users the effort of needing to install both Retrospect itself, run it in order for all the channels to be deployed as add-ons, and then again install all channel updates. Besides this, if we would not update the main add-on it will contain outdated channels. The main risk then is that a user installs the main Retrospect add-on (with outdated channels), then installs all channel updates from the same list because he/she sees them in the main listing (see picture) and only then runs Retrospect. Retrospect will then automatically deploy the old channel updates over the new updates (mental note to self….I should perhaps prevent this).

So thing to remember is: when starting clean there is only the need to install the main add-on as it will include all recent updates…..BUT….the details are in the “starting fresh“. Because what is starting fresh? Uninstalling Retrospect, or deleting all folders manually and the install? Remember the part where Kodi caches packages in the addons/packages/ folder?

“Starting fresh” without clearing that cache will cause an old package from the cache to be installed.

That is why the steps in our troubleshooting guide specifically mentions to clear the addons/packages/ folder before reinstalling.

Throttling

A final thing. What happens if the add0ns.xml.md5 is updated? Within 1o minutes (as Kodi checks it every 10 minutes) all online users will get a notification that a new update is present. And all those users will start downloading it. That seems ideal does it not? Well, no, it doesn’t. With 120.000+ users, roughly 50.000 will start download an update within the first 10 minutes (and that is a low estimation). With about 1 MB download size, that means that my poor download server needs to process 50 GB of data within 10 minutes and it does not like that….at all. We had a number of server outages because of this and were even taken offline by our hosting party due to NIC saturation.

So the solution that is implemented is to throttle the downloads. To make it simple we do this based on the IP address from which the files are being requested. Some IP’s get the updates first, other later. This period is spread over about 6 hours time. That way, that 50 GB (or more) is spread over those 6 hours and all the server stay happy.