Web Crawler/Spider for NodeJS + server-side jQuery ;-)

JavaScript6617mit

4 months ago

cheeriocrawlerextract-data

A scalable, mature and versatile web crawler based on Apache Storm

HTML841apache-2.0

2 months ago

apache-stormcrawlerdistributed

A high performance web crawler / scraper in Elixir.

Elixir915

7 months ago

crawlerelixirfiles

Run a high-fidelity browser-based crawler in a single Docker container

JavaScript500agpl-3.0

2 months ago

crawlercrawlingwacz

The archivist's web crawler: WARC output, dashboard for all crawls, dynamic igno

Python1233other

3 months ago

archivingcrawlcrawler

Wget-compatible web downloader and crawler.

HTML523gpl-3.0

6 months ago

Roguelite dungeon crawler game

C++195gpl-3.0

5 months ago

Privacy Web Search Engine (not meta, own crawler)

C++58agpl-3.0

10 months ago

cppcrawlerencryption

Incredibly fast crawler designed for OSINT.

Python10503gpl-3.0

4 months ago

crawlerinformation-gatheringosint

The unix-way web crawler

Go216mit

3 months ago

clicrawlergo

A simple but powerful web crawler library for .NET

C#229mit

4 months ago

crawlerrobots-txtspider

Play with Google Play API :)

Java551other

9 months ago

A Powerful Spider(Web Crawler) System in Python.

Python16128apache-2.0

10 months ago

crawlerpython

A scalable web crawler framework for Java.

Java11022apache-2.0

5 months ago

crawlerframeworkjava

Lightweight Ruby web crawler/scraper with an elegant DSL which extracts structur

Ruby1300mit

3 months ago

crawlerdslruby

Elegant Scraper and Crawler Framework for Golang

Go21293apache-2.0

5 months ago

crawlercrawlingframework

A self-hosted BitTorrent indexer, DHT crawler, content classifier and torrent se

Go1908mit

6 days ago

bittorrentdhtprowlarr

Pagser is a simple, extensible, configurable parse and deserialize html page to

Go94mit

6 months ago

collycrawlerdeserialization

brozzler - distributed browser-based web crawler

Python618apache-2.0

3 months ago

Description Shattered Pixel Dungeon is a traditional roguelike dungeon crawle

Java4313gpl-3.0

last month

androidgamegame-development

TypeScript31

2 years ago

Appium Native Crawler CLI - Features include: Screenshots, Performance, Accessib

Ruby47apache-2.0

5 years ago

[Crawler/Scraper for Golang]🕷A lightweight distributed friendly Golang crawler

Go210apache-2.0

4 years ago

crawlergogolang

make mirror of https://packagist.org

PHP55cc0-1.0

2 years ago

A Python DNS crawler to find identical domain names under different TLDs.

Python20other

5 years ago

crawlercybersecuritydns

An easy-to-use and highly customizable crawler that enables you to create your o

Scala24mit

7 years ago

A first person dungeon-crawler for DOS, written in C++, using software rendering

C++44bsd-2-clause

2 years ago

djgppdungeondungeons

Open Source Web Crawler for Java

Java4455apache-2.0

2 years ago

a dungeon crawler written in TypeScript using React and svg

TypeScript65mit

7 years ago

Nov 20 2017 -- A distributed open source search engine and spider/crawler writte

C++1481apache-2.0

2 years ago

Ragpicker is a Plugin based malware crawler with pre-analysis and reporting func

Python90

9 years ago

Squidwarc is a high fidelity, user scriptable, archival crawler that uses Chrome

JavaScript162apache-2.0

4 years ago

browser-automationchromechrome-headless

Voight-Kampff is a Ruby gem that detects bots, spiders, crawlers and replicants

Ruby176mit

last year

Astray is a lua based maze, room and dungeon generation library for dungeon craw

Lua136zlib

7 years ago

dungeon-crawlerlove2dlua

:vertical_traffic_light: Block malicious crawlers with Cloudflare Firewall Rules

190mit

4 years ago

cloudflarecloudflare-firewall-rulescrawler-detector