▲ ▼ Link rot, Invalid web URLs

web
archive

posted by Abishek Muthian Ideator , 1110 days ago permalink , show insights

Link rotting is when a particular URL is not valid anymore, Usually because the content has been removed. Link rot is an issue we face with needgap as the majority of links here are of side-projects and when the project unfortunately ceases to exist the links become invalid.

If the links are automatically archived then it can help the community can understand what it offered and perhaps even help understanding why it didn't succeed.

Of course link rot is not specific to needgap, Whole wide web is affected by the link rot.

WHO NEEDS THIS

▲ ▼ TobiasStarling , 1104 days ago permalink

You should look at link rot and NTFs! I can see that being a serious problem that would have some serious value if you can solve it and explain it to people properly

reply I WILL PAY FOR THAT
HOW MANY WILL PAY FOR THIS
▲ ▼ Abishek Muthian Ideator , 1103 days ago permalink

Oh that explains why this need-gap has higher than average views than most new need-gaps. Thanks for bringing it to me.

I'm not into NFTs, I presume the URL to the actual content stored on the NFT becomes invalid? IPFS should address this as long as peers exist, But just as archive.org there could be need-gaps to make this process seamless and accessible.

reply I WILL PAY FOR THAT
HOW MANY WILL PAY FOR THIS
▲ ▼ Abishek Muthian Ideator , 1110 days ago permalink
Services like archive.org saves web pages among other things and help address link rot directly. But saving individual pages protected by captcha are not scalable and to address this archive.org provides couple of solutions:
- ArchiveBot; An IRC bot which saves entire website upon request.
- API; To programmatically archive content.
I've used ArchiveBot to save entire websites with hundreds and thousands of URLs before and yet to try the API to address the aforementioned problem.

Even if URLs are archived in real-time, Identifying and replacing the URL when it's not accessible anymore requires additional work. So there's a need for for an end-to-end solution for identifying and addressing link rot.
reply I WILL PAY FOR THAT
HOW MANY WILL PAY FOR THIS