Commit Graph

81 Commits (d42bf5a0310b8df4babff645ee91c37e9f994bfe)

Author SHA1 Message Date
erdgeist d42bf5a031 ** struct ot_workstruct gets ritcher (and will become even ritcher soon).
This is where we encapsulate all per-request data from peer to hash to peer_id, so that it is
available everywhere without passing hundreds of pointers down the stack. Most functions that
do work down the stack now accept an ot_workstruct and some flags. So it can end up in the
stats/event-handler where it will be the default parameter in the future.

** peer_id is now being copied by default and moved to ot_workstruct
So it is available in stats and subsequent functions.

** sync scrape madness is gone
SYNC_SCRAPE was intended to sync tracker state that would normally be lost on restarts i.e.
downloaded counts per torrent. The way was to push it in the tracker cloud after finding all
neighbouring trackers.
This is madness. It never was tested and can be done per tracker by fetching
stats/mode=statedump from time to time and starting opentracker with the -l option later.

** livesync thread has its own ot_workstruct now
So it can behave like ot_udp and ot_http against trackerlogic.c and get rid of the first half
of the embarrassing global variables. The sending half will be fixed soon [tm].

** stats can log completed events
The author recognizes the needs of original content distributors to keep track of the amount
of times a work has been downloaded. While not feasible and used on openbittorrent and other
open and anonymous tracker installations, a tracker user can now choose to send those events
to syslog.
15 years ago
erdgeist 90e7262d9d Preparing more efficient header parsing 15 years ago
erdgeist 478884660f Enable limiting fullscrapes to only every 5 minutes per IP 15 years ago
erdgeist 6c51fca9a1 Introduce the concept of an ot_net into opentracker. 15 years ago
erdgeist d113912101 Log networks in its old form has been disabled. Live logging may come back soon. 16 years ago
erdgeist c7ed890222 Fix white spaces
Introduce loading tracker states with -l
Alter tracker state to a human readable form
16 years ago
erdgeist 9297967f85 Add a stat option the count bucket stalls
Add an iterator for all torrents. It's slower but for stats it's okay.
Move some stats collection stuff to the new iterator. More to come.
Start a "report all stats" page.
Start fixing the code to identify "busy" networks.
Add the concept of hosts allowed to proxy.
Add a parser for the X-Forwarded-For: HTTP header.
Clean up HTTP Header handling code. (Remove some left overs of now vanished sync code).
16 years ago
erdgeist a6fe338040 Remove superfluous test for == NULL, when variable has been used, anyway 16 years ago
erdgeist f12fa4c362 Use a self pipe to wake from io_wait when a thread finished his work. 16 years ago
erdgeist 930495a0db We sent an incorrect reply dictionary vor v6 announces 16 years ago
erdgeist 131211b4da V6 16 years ago
erdgeist 779d6c235f * http and udp routines now use thread local buffers passed in workstruct containers. In other words they do not use static_buffer anymore and are considered to be thread safe.
* the new workstruct also introduces a well defined buffer and result passing path
* a new function scan_find_keywords is a wrapper around scan_urlencoded_query that maps keys in url to values passed in an array of ot_keywords structs
* this new function cleans up much of url parameter parsing work, where read_ptr and write_ptr have been introduced rather than the confusing char *c, *data variables
* I now use memcmp instead of byte_diff to allow compiler to optimize constant size string compares
* got rid of UTORRENT_1600_WORKAROUND
* livesync_ticker is now only called from one (currently main) thread to avoid race conditions
16 years ago
erdgeist 2df09905f5 * opentracker now drops permissions in correct order and really chroots() if ran as root
* lock passing between add_peer_to_torrent and return_peers_for_torrent is now avoided by providing a more general add_peer_to_torrent_and_return_peers function that can be used with NULL parameters to not return any peers (in sync case)
* in order to keep a fast overview how many torrents opentracker maintains, every mutex_bucket_unlock operation expects an additional integer parameter that tells ot_mutex.c how many torrents have been added or removed. A function mutex_get_torrent_count has been introduced.
16 years ago
erdgeist a9ab9b0c0d Make all torrents in their buckets sorted again. 16 years ago
erdgeist c6947b160f Handle program end more politely 16 years ago
erdgeist 08c7162783 Renamed OT_FLAG to OT_PEERFLAG to make code easier to read
Introduced READ16/32 and WRITE16/32 makros to abstract loading/storing from unaligned addresses away on cpu's that can actually load/store everywhere
Removed all unnecessary memmoves, especially where it only moved 6 bytes in inner loop. I replaced them with WRITE16/32(READ16/32()) makros
16 years ago
erdgeist e2199cc504 New peer flag to see, whether peer came from announce or sync 16 years ago
erdgeist 566e8267e1 Live Sync for peers that do not come back too early. 16 years ago
erdgeist 334c6e4bbb The BIG refactoring [tm]. Too many changes to count them. If it doesn't suite you, revert to last version. 16 years ago
erdgeist daaee855b2 Whitespace fixes 16 years ago
erdgeist 206c74ee2f Make ot_try_bind local again. 16 years ago
erdgeist e534db03c6 added live sync code
added a config file parser
added tracker id
changed WANT_CLOSED_TRACKER and WANT_BLACKLIST into WANT_ACCESS_WHITE and WANT_ACCESS_BLACK
changed WANT_TRACKER_SYNC to WANT_SYNC_BATCH and added WANT_SYNC_LIVE
added an option to switch off fullscrapes

cleaned up many internal hardcoded values, like PROTO_FLAG,
16 years ago
erdgeist c28cf398fd Introduce some kind of versioning 17 years ago
erdgeist 848a06a706 Drop ot_{byte,word,dword} and use uint{8,16,32}_t, also simplify includes 17 years ago
erdgeist c5e5569770 Fixing mmap allocation went to ot_mutex.c. Clean all torrents moved to ot_clean.c. Also make static buffers static 17 years ago
erdgeist 697e239027 Moving accesslist stuff to its own object 17 years ago
erdgeist a0d12f9ab5 Full scrape moves to its own object 17 years ago
erdgeist 41120f6a0d Move stats out of opentracker.c, also have an own file handle udp requests. 17 years ago
erdgeist 8900cc0dd9 No one can get access to buckets now without locking them. Also split up the trackerlogic.c-monster in functional sub-units. HEADS UP: this code is untested and not considered stable. 17 years ago
erdgeist 785a9f13bd Make to_hex thread safe. Get rid off old /24 counting code. Protect more bucket accesses by locks. 17 years ago
erdgeist 4a68f8692b We do now use the tools for locking buckets in order to prepare MT code 17 years ago
erdgeist a6b8311812 Introducing first tools to make opentracker multithreaded. 17 years ago
erdgeist 8acc0ff48f Introduce configuration dependent number of parameters in a nicer way 17 years ago
erdgeist 853dc33be2 Allow for more than one admin ip address 17 years ago
erdgeist dc8e889797 Add more full scrape stats 17 years ago
erdgeist 31eada6168 Reenabled syncing 17 years ago
erdgeist b19bbd6a85 Number of buckets is 1024 now
Clean all torrents now only cleans one bucket at a time
All torrents that are being worked upon in an announce are being cleaned on demoand
torrent's peer lists now keep extra counts for seeds and peers to speed up scrape and announce
Sync has gone for now. I will think up a new way to implement. The old one was way to slow.
17 years ago
erdgeist 41019c79d8 Introducing multiscrape 17 years ago
erdgeist daf79f694d After studying opentracker live data, I decided to radically reduce default allocation for vectors and making it grow faster instead: Most pools only had one or two peers, wasting 8*15 or 8*14 bytes. 17 years ago
erdgeist b8b1493127 Increase check interval for timeouted sockets to 10 seconds. That may prevent some disrupted scrapes. 17 years ago
erdgeist be117f96a2 Increase send timeout. Also use an own variable for that. 17 years ago
erdgeist 6c1adb8fc8 since gettimeofday is rather expansive, we do only fetch time once in a 5 second period, when we are delivered a SIGALRM. 17 years ago
erdgeist 4c15dc7a70 make 64bit safe 17 years ago
erdgeist c350fa0b3c New stats for s24s code, this is debug, do not use in real world 17 years ago
erdgeist 33c9c530d0 An announce with event=stopped now returns correct number of leechers and seeders. In TCP and UDP. 17 years ago
erdgeist 3e47339b6c Added whitelisting to reimplement the WANT_CLOSED_TRACKER functionality 17 years ago
erdgeist afe966b644 some subnet stats... still ugly and untested. 18 years ago
erdgeist c0b00c0bf5 * fixed "nobody " -> "nobody" fuckup when getpwnam-ing
* implemented basic blacklisting:
** the file specified with -b <BLACKLIST> is read and added to a blacklist vector
** if an announce hits a torrent in that blacklist vector, add_peer_to_torrent fails
** sending a SIGHUP to the program forces it to reread the blacklists
** the server returns with a 500, which is not exactly nice but does the job for now
** an adaequat "failure reason:" should be delivered... TODO
18 years ago
erdgeist 33774078ab Added inbound part of sync. This also meant to remove the black/whitelisting as we did by now. A more scalable way to blacklist will follow. 18 years ago
erdgeist b38104b986 Added outbound part of sync Proposed format: d4:syncd20:<info_hash>8*N:(xxxxyy)*Nee, therefore had to refactor torrent cleanup now that it will hit all torrents once every OT_POOL_TIMEOUT units. 18 years ago