Dropcatch+ | Filter Daily Lists of Expiring Domains

[Croc]

I wrote a little Python script to aid myself in filtering daily expiring domain lists by criteria I specify. I have it set to run automatically using the built-in Automator application on macOS, but it can easily be run with a UNIX cronjob or Windows Task Scheduler as well.

You can review the full source code on Github and download the latest release here.
If you are knowledgeable in Python, contributions are welcome. Just make a pull request anytime and I'll review it as soon as possible.

Comes with a simple JSON configuration file that looks like this:

{
  "maxDomainLength": 10,
  "keywords": [
    "minecraft",
    "coin",
    "craft",
    "tech",
    "pvp"
  ],
  "tlds": [
    "com",
    "net",
    "org",
    "io",
    "co"
  ]
}

The script programmatically downloads the next day's expiring list from Namejet's website and parses it. It then filters the entire list in mere seconds based on the criteria you specify in the config.json file.

If running on a timer of some sort, please set the script to run AFTER 7:00 AM PST because the next day's list is not available until that time the day before.

Installation Instructions

1. Download the Python 3.6.x+ installer from https://python.org/downloads
2. Run the installer, but please make sure to tick the box labeled "Add Python to environment variables". It is unchecked by default. This is important!
3. Open Terminal (Mac/Linux) or Command Prompt (Windows) and type the following command to let the terminal know where the script files are located. Replace [file path] with the full path to the directory containing the files: cd [file path]
4. Next, run the following command: pip install -r requirements.txt
5. The above steps only need to be run once. Now every time you want to run this utility, you simply run the following command: python main.py (if that doesn't work, try replacing python in that command with python3)
6. If executed correctly, you will see domains being output according to your criteria and they will also be saved to a file in the results directory with the name results_MM-DD-YYYY.txt

 

[MarkFreak]

I made a quick pull request and an issue, and also you should make a white version theme for your threads ;/

 

[Croc]

I made a quick pull request and an issue, and also you should make a white version theme for your threads ;/

Merged for now, but I'm thinking about reverting it because you said the F-strings were only just added in Python 3.6 and the initial version of the script works with all versions of 3.x.x

Do you think the majority of Python users are using the up-to-the-minute latest version? It might be better to keep it somewhat compatible with a few prior releases.

 

[MarkFreak]

Merged for now, but I'm thinking about reverting it because you said the F-strings were only just added in Python 3.6 and the initial version of the script works with all versions of 3.x.x

Do you think the majority of Python users are using the up-to-the-minute latest version? It might be better to keep it somewhat compatible with a few prior releases.

If you're going for compatibility use .format. Python 3.7 is coming soon and Python 3.6 has been out for a while, so I think it shouldn't be a big problem for them to update. I'd stick with fstrings as they are just really cleaner and have their use, you can read the PEP I linked you.

 

[Croc]

Just pushed a critical fix for a bug on Windows systems that prevented the script from running properly.
Download the latest zip from Github here.

Thanks to Niko Bellic for reporting the bug.

 

[wavy]

Traceback (most recent call last):
  File "main.py", line 19, in <module>
    config = json.load(fp)
  File "C:\Users\mitch\AppData\Local\Programs\Python\Python36\lib\json\__init__.py", line 299, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "C:\Users\mitch\AppData\Local\Programs\Python\Python36\lib\json\__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "C:\Users\mitch\AppData\Local\Programs\Python\Python36\lib\json\decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Users\mitch\AppData\Local\Programs\Python\Python36\lib\json\decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 7 column 3 (char 109)

ermm, what did I do wrong?

 

[MarkFreak]

Traceback (most recent call last):
  File "main.py", line 19, in <module>
    config = json.load(fp)
  File "C:\Users\mitch\AppData\Local\Programs\Python\Python36\lib\json\__init__.py", line 299, in load
    parse_constant=parse_constant, object_pairs_hook=object_pairs_hook, **kw)
  File "C:\Users\mitch\AppData\Local\Programs\Python\Python36\lib\json\__init__.py", line 354, in loads
    return _default_decoder.decode(s)
  File "C:\Users\mitch\AppData\Local\Programs\Python\Python36\lib\json\decoder.py", line 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "C:\Users\mitch\AppData\Local\Programs\Python\Python36\lib\json\decoder.py", line 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 7 column 3 (char 109)

ermm, what did I do wrong?

Your config is invalid JSON. You can post your config here so I can take a look at what you did wrong or you can use an online linter.

 

[Croc]

ermm, what did I do wrong?

JSON can be very picky, so you probably forgot a comma when you edited the config file.

Either MarkFreak or I can assist you with that if you post the config.

 

[wavy]

JSON can be very picky, so you probably forgot a comma when you edited the config file.

Either MarkFreak or I can assist you with that if you post the config.

I did accidently add a comma, thanks!