×

32
33

Tool that collects channel lists, VOD and series details from providers, and some initial dataDiscussion (self.IPTVGroupBuy)

submitted by DrMantisTobbogganValued Collaborator

Inspired by /u/TorgoJohnson 's post, I made a little script that will scrape channel, VOD and series details from one or more providers and save them to CSV files.

I'm hoping this will help in comparing providers, and help folks who are looking for specific channels to check whether they're present in a provider before trialing a service.

The code is on GitHub.

I have uploaded CSVs for the services I have access to on Google Drive. If someone wants to copy this to the main Google Drive for provider comparisons, let me know.

The services I've collected details for are:

  • Eagle 4K
  • Strong 8K

The data I've collected is very raw. I can easily tweak the code to output to a Google Sheet to make it more consumable. Let me know what would be useful.

I also got trials for Trex from two different sellers on Z2u. Interestingly, there were some pretty big differences in the number of live channels and movies in particular. There's also differences in channel logo URLs. I haven't done a detailed comparison to see if this makes a difference in real usage.

I did a second run to add a third Trex trial. This found some interesting things:

  • Sellers 2 and 3 have identical live channels. Seller 1 has more channels.
  • From the first and second runs of the script, about 10 mins apart, the amount of VOD content for seller 2 increased substantially and was by the end, more than either seller 1 and slightly less than seller 3.

I wonder if this is just updates gradually rolling out across different sellers. I thought it was interesting anyway so thought I'd share.

all 28 comments
sorted by:
best

[–]Not-Not-Maybe 2 points3 points  (3 children)

Wow, I’ve been wishing for a tool like this. Thanks for creating it!

Edit to add: I read the readme, and I don’t know anything about how to use Python so I probably won’t use the tool, but I appreciate you sharing the CSV files in the google drive. Thank you very much.

[–]DrMantisTobbogganValued Collaborator[S] 0 points1 point  (2 children)

I’m glad you like the idea. The tool is a bit rough around the edges still. There’s a couple of things I want to add to it first but then I can make it easier to run.

Is it more useful to you to have a tool that collects this data for your own provider(s), or to have the data I published in an easier to use format eg. a Google sheet or some kind of web interface?

[–]Not-Not-Maybe 1 point2 points  (1 child)

Looking at data in an easy-to-sort/filter format for a variety of z2u/Alibaba providers is my end goal. But I wouldn’t want you to have to do extra work to make that happen. I really appreciate what you have done already. 😊

[–]DrMantisTobbogganValued Collaborator[S] 1 point2 points  (0 children)

That’s one of my goals too once I get the basics sorted out. I’d love some suggestions on how to present this data so it’s easy to sort and filter and compare across providers, or if you’re up for building that, I, and I’m sure the rest of the community here, would really appreciate it.

[–]jcumb3rValued Collaborator 2 points3 points  (1 child)

I just ran this for the first time tonight and it's incredibly simple and effective. Thanks again for putting this together. I added channel lists for Dino & Lion to the shared folder with the rest of the channel lists you provided.

[–]DrMantisTobbogganValued Collaborator[S] 1 point2 points  (0 children)

Thanks. I’m glad it’s useful.

[–]great235 1 point2 points  (0 children)

Sweet! Thanks! Definitely makes picking the right service that much easier.

[–]TorgoJohnson 1 point2 points  (3 children)

Terrific Work. I am going to save the files to my hard drive and take a closer look. On first glance Trex has some movies not available with the other services and also has my local CW station. Only Mega has my local PBS station.

[–]DrMantisTobbogganValued Collaborator[S] 0 points1 point  (2 children)

Awesome. I’m interested to hear about what you discover.

[–]TorgoJohnson 2 points3 points  (1 child)

An oddity. Corner Gas ran for six seasons, was popular in Canada and even was shown in the United States and it is not on any service even Trex. Ultimately, I decided just keep my current services Strong 8K and Plex. I am on a budget and getting three seems like overkill at this time. I have a Easynews sub for 2 bucks a month and I find can any older movie or TV show there.

[–]DrMantisTobbogganValued Collaborator[S] 1 point2 points  (0 children)

Yeah I still rely on the Usenet/arrs/Plex combo for anything VOD. I can almost always find what I’m looking for with that. I still need to give Strmio and Real Debrid a proper go. Funny you should mention Corner Gas. That’s next up in my watch list.

[–]jcumb3rValued Collaborator 1 point2 points  (1 child)

This is awesome, thanks for creating the code! I've added a link to your folder in the main Google drive mentioned in the sticky post. Any updates you make within that folder will stay in sync there. Thanks again!

[–]frettbe 0 points1 point  (0 children)

I have Dream4K and 4KOTT if you want to add them. Let me know how I can send it to you

[–]dunkelman13 1 point2 points  (0 children)

Working just fine

<image>

Thank you so much. Great tool, great job!

[–]trixr1907Newbie 2 points3 points  (1 child)

a wonderful tool, great work i appreciate it. i am currently creating a csv of all german channels from a strong8 provider. maybe it is relevant for someone?

[–]AutoModerator[M] 0 points1 point  (0 children)

This post was flagged for mod review because of a lack of positive history on Reddit. If it does not violate our rules, it will eventually be approved.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[–]KidRockzValued Collaborator 1 point2 points  (6 children)

Is it possible to add Channel resolution (720p, 1080p, 2160p) in two columns - one column that will list the Labelled resolution and another column to list the Actual resolution (Labelled resolution maybe 4K but actual resolution might only be 1080p)?

[–]DrMantisTobbogganValued Collaborator[S] 4 points5 points  (5 children)

I can get the labelled resolution but it looks like the script will need to make an API call for each stream so it will take much longer to run. I'm happy to try it though. I haven't found detailed documentation for the xtream API yet so there may be a simpler way than that. I'll see what I can work out.

To get the actual resolution will require loading each stream I think, so slower again. I'll give it a go though because it seems like it's pretty valuable. As a bonus, this would also tell us which channels are offline.

[–]sickpup93 1 point2 points  (0 children)

If you give that a go and it's not much more work, the audio could be beneficial as well. Whether it's stereo or 5.1. Good work!

[–]gusestrellaValued Collaborator 1 point2 points  (3 children)

I believe you are correct. I get resoultion and framerate with ff_probe (part of the ff_mpeg utilities). I have a small python script that you can specify specific group to get this data (ex. groups I care about) or channels (ex espn ).

I love the idea of saving tehse into searchable files, so then can slo look into changes over time - what providers change channels and so on. Also how often movies being added or series. Can even do things like delay from series have aired and how long providers take to add them.

I also been playing around with epg data and get sense how manay days of data available and so on .

Wil check your code but very interested on this type of utility.

[–]DrMantisTobbogganValued Collaborator[S] 0 points1 point  (2 children)

Ah interesting. Sounds like you’ve got further than me with some of this. If you’re able to share your script, or contribute a pull request adding getting more stream details. I’d really appreciate it. I’m also up for any other contributions.

One of the ways I’d like to use this is as you say, to track changes in provider offerings over time. To do this I plan to run the script in a cron job and dump the results into Bigquery for further analysis and/or another DB for presenting through a web interface.

Those are all very loose plans though. For now, I’d like to focus on getting the raw, fundamental data captured while I get my head around the problem space. That said, I’m very supportive of you, or anyone else, running with this in whatever way you like.

[–]gusestrellaValued Collaborator 1 point2 points  (0 children)

just placed the code that identifies channel resolution and frame rate on GitHub. Check it out at https://github.com/estrellagus/iptv-tools. ping me if have question but should be very easy to see the two routines to lift very easily to get this functionality.

[–]gusestrellaValued Collaborator 0 points1 point  (0 children)

I will create a GitHub with the script I have next week and share your way. I have other misc tools too around epg that figure can help folks or other can take and further enhance.

An idea that though of would be useful o save the details from the user info end point. Can be of us seeing things like the message or url (can mask the server and other critical data like below

{
    "user_info": {
        "username": "XXXXX",
        "password": "YYYYY",
        "message": "Welcome to World UNKNOW",
        "auth": 1,
        "status": "Active",
        "exp_date": "1763357605",
        "is_trial": "0",
        "active_cons": "0",
        "created_at": "1731821604",
        "max_connections": "1",
        "allowed_output_formats": [
            "m3u8",
            "ts",
            "rtmp"
        ]
    },
    "server_info": {
        "url": "UUUUU.cdngold.me",
        "port": "80",
        "https_port": "443",
        "server_protocol": "http",
        "rtmp_port": "25462",
        "timezone": "Europe/Amsterdam",
        "timestamp_now": 1735960020,
        "time_now": "2025-01-04 04:07:00",
        "process": true
    }
}

[–]Averexs 0 points1 point  (2 children)

The Github page seems to have disappeared.

[–]DrMantisTobbogganValued Collaborator[S] 0 points1 point  (0 children)

Ah sorry. I was cleaning some things up and accidentally removed it.

I’ll put it back. Thanks for the heads up.

[–]DrMantisTobbogganValued Collaborator[S] 0 points1 point  (0 children)

It should be back now.

[–]BigTurboChungus 0 points1 point  (0 children)

Hey bro, just saying do you want a trial to test of trex with all the channels? I also have the mega dealer panel too if you want a trial with all channels of that

Edit: because I know they don't give you all the channels with a trial but I will for the sake of this group