A fully featured provision script, based on Python (FREE) | v0.8.2a

Status
This thread has been locked.

Frank

Applications Admin III at Ferguson Enterprises
Banned
Feedback score
30
Posts
697
Reactions
402
Resources
0
Introduction

Hello everyone,

Long time no see? I'm most commonly known as being the Chief Systems Engineer/Chief Technical Officer at PebbleHost. I do not work there anymore, and have broadened my career beyond minecraft. With that being said, I see that there's a lot of people who still require assistance with basic provisioning of servers. Particularly, with nginx, php-fpm, pterodactyl, mail servers, firewall configurations, ect... As a result, I've decided to work on a script that is designed to be a AIO solution to these problems.

This will be provided free-of-charge to everyone and posted on github once completed.

Important Update:

Due to the complex nature of the script, I'll be migrating the script from bash to python entirely. The direction that the script is wanting to go will not be reasonable to carry out such a large script with exclusively bash. However, the script will remain open source. Please bare with me while the rewriting process is completed.

Note: Only CentOS 8 will be available upon release.

Warning:
This script is not a replacement for typical system administration management, and should only be used for initial provisioning of your server. It serves as a easy-to-use interface to make the installation and configuration process as painless as possible, along with bringing proper security practices to light and in a way that's easily obtainable without having to have the understanding of the working of the script. While this script can be used on pre-existing servers, it is recommended to not use the automated installer as it WILL overwrite existing installations.
Paid support can be found here.
To be clear, any problems and updates to the script will be provided free of charge.
Requirements:
- Server running one of the supported distros, which include
- CentOS 7
- CentOS 8
- Debian 8
- Debian 10
-
Ubuntu 18.04 LTS
- Ubuntu 20.04 LTS.
- A working internet connection
- Functional DNS resolution.
- Approximately 10-30 minutes, depending on system specs.

- Disk Space is variable depending on what you wish to install.​
Recommendations:
- At least a 1280x720 (720p) terminal window, or terminal size of 132x43
- SSD/NVMe server to accelerate installation processes; script was not tests on HDDs
- A separate terminal tab/window to switch to during installation for other tasks, if needed
- A reboot post-installation/provisioning wouldn't hurt, but is not required.

Features:

Not all features available on release. AOR = Available on Release
  1. User-friendly windows-like installation process - Completed, AOR
  2. Robust logging capabilities, presented to the user in a easy-to-read fashion - WIP, AOR
  3. Multi-distro support - WIP, AOR w/limitations
  4. Interface Selection menu to select which interface you would like the installation to take place - Completed, AOR
  5. Automatic error detection and abort if fatal error is detected. - WIP, AOR
  6. Automatic NGINX Configuration Wizard - Completed, AOR
  7. Automatic Pterodactyl Installation Wizard - WIP, AOR
  8. Automatic Installation Wizards for commonly installed programs/software, including but not limited to; docker, mariadb, multicraft, tcadmin, wireguard VPN, OpenVPN - WIP, AOR with limitations
  9. Automatic Installation of intuitive docker containers like: poste.io, mailcow, and others - WIP, AOR
  10. Automatic firewall configuration with IPSet rules, and general server hardening for common attack vectors like SSH brute force - Completed, AOR
  11. Free list of known compremised, or otherwise bad reputation IP addresses automatically blocked on a software-level, comprising of over 3,500,000 IP addresses. - PP (Postponed)
  12. Selection to only permit proxied connections to connect to the server via any port of your choosing, for instances like cloudflare spectrum, or other 3rd party services - WIP
  13. Automatic detection of last connected IP address, allow this address by default on firewall to port 22/tcp - Completed, AOR
  14. Intelligent & Feature Packed Log Scanner & Notifier to scan common log files like dmesg/messages, nginx, mariadb/mysql, docker, and several others - and notify you of potential issues according to these logs via email. - PP (Postponed)
  15. Standard Cleaning protocols like periodically by cleaning /tmp, and keeps the system tidy of needless files - WIP, AOR
  16. Automatically set permissions for maximum security in a multi-user setup - WIP, AOR
  17. Wizard to create users with different user profiles ranging from Standard Access (chroot jail to their directory, with only SFTP access) to Escalated Access with full sudo access - WIP, AOR
  18. Implicit ACL rules to aid in security hardening, with full wizard configuration process - WIP, AOR
  19. Intelligent automatic repair tool to attempt automatic repairs for common problems, with logging of all commands and methodologies used to achieve fix. If these fail, present the user with the output of all commands and the commands ran, and attempt to revert changes - Not Started
  20. Automatic backup script & Automatic update script - Not Started
  21. Common System Administrator tools installation for quality of life, such as nload, dstat, htop, asn, and others, including custom scripts - Completed, AOR w/limitations
  22. Intuitive UI to configure firewall rules with robust capabilities like filtering based on IP address or destination address to a specific port - WIP, AOR

Documentation (WIP)

All information is logged within /var/log/manager.log. You will be able to find exit statuses here presented in a easy-to-read fashion with timestamps. Below are known error status, what they mean, and what a possible fix is. Your logs will look something like this:

2Fcube02AwkT.png

Each function will return either one, or several exit statuses. Each section of a function will return it's own exit status. This is to aid in adding the most verbose logging, without having to sift through the logs with grep. Additionally, for an ease of use of grep, if needed, a date and time stamp are added for precise viewing at relative ease, and will show when each function is initialized, and where each subfunction is initialized, to know exactly where the problem occurred. Each exit status is classed with different levels.

Green - Level 0 - OK, normal
Yellow-Green - Level 1 - OK, modifications needed
Yellow - Level 2 - Normal Operation, but cannot continue
Orange - Level 3 - Non-fatal error, but should be investigated
Red - Level 4 - Fatal error, script aborted. Refer to documentation
Purple - Level 5 - Fatal error. Unknown error. Systems Admin needed.
Exit Status: 0 - Successful. This is no reason for concern.
Exit Status: 2 - Successful. Modifications Required. This is used solely for script logic while installing on CentOS 7.
Exit Status: 4 - The script has detected it ran before and cannot be ran again. You should read the information given within ~/manager.info for more information.
Exit Status: 22 - A function failed a self-check. This can happen if not all the packages needed were installed, or a service failed to start.
Exit Status: 10 - A package manager could not be found, or was not known. This error occurs when the distro you are using the script on is not supported or known.
Exit Status: 255 - An unknown error has occurred and will require an investigation with an experienced system administrator.

Error handling will look something like this:
mLO5Rkc4kAC3.png


VRbVX2VwSZde.png


Additional Information
Due to the nature of this script, it will be very large and will take awhile to develop fully. Beta testing may be available, however this will only be strictly recommended for development servers. For full release, you can use this script in regular deployments.

Current Version: 0.7.3

Change Log 0.7.0 -> 0.7.3
- [+] Broke Pterodactyl Installation into 5 distinct phases for legibility and debugging purposes
- [+] Added a function systemUpdate to call anywhere in script for dynamic system updating for streamlining
- [+] Added adjustable variables if desired by user in dedicated section of script, or on runtime
- [+] Dynamically generated passwords with 16 characters where passwords are required
- [+] MariaDB/MySQL installation is now it's own function
- [+] Updated system requirements on thread for users to lookover
- [-] Removed distro specific commands
- [?] Begin work on documentation on gitlab project
- [-] Removed mailcow support on CentOS 8 due to compatability issues

AUTOMATIC REPAIR TOOL
Version 0.0.1
- [+] Added user confirmation with information below​
The automatic repair tool is a separate script from the manager script that provisions your server and installs the above software, automatically, with minimal to no input from the user, presented in a easy-to-use, intuitive GUI.

This script attempts to automatically repair any problems detected on your server by using log files and extrapolating from the information given through the log files, and of commands ran using advanced methodologies and attempt to emulate a problem-solving critical-thinking thinking pattern. It is worth noting that the script WILL modify low-level configurations, and while these configurations WILL be backed up before any modifications are made, as a result of these modifications, it is possible that the script can potentially break some things. If this is detected by the script, all changes will be reverted automatically and you will be presented with an overview of what the script did with the full logs. There is no GUI for this program. Not all problems will be detected, and not all problems may be fixed using this script.
 
Last edited:
PebbleHost
High performance, consistent uptime and fast support. Minecraft hosting that just works.

Anish

Developer
Supreme
Feedback score
9
Posts
365
Reactions
176
Resources
0
This looks sick! Can't wait for it to come out. (If you want to make a profit maybe offer support for this open source version?)
 

Frank

Applications Admin III at Ferguson Enterprises
Banned
Feedback score
30
Posts
697
Reactions
402
Resources
0
Why would you blacklist so many IPs instead of just whitelisting the ones you want to allow?

By default, the rule is to drop the packets, however for certain ports, such as 25565, you can't really (or at least reasonably) whitelist only certain IPs, especially if it's a public server.

The rules are meant to avoid exploitation from known compromised, or otherwise bad reputation IP addresses.
 
Banned forever. Reason: Rules violations

JaGv

Operator | OneShot MC jag-dev.github.io
Premium
Feedback score
1
Posts
48
Reactions
24
Resources
0
These seems like an awesome tool for system administrators. Maybe a future version could include Apache/Web Server installation and setup.

Can't wait until this comes out!
 

Frank

Applications Admin III at Ferguson Enterprises
Banned
Feedback score
30
Posts
697
Reactions
402
Resources
0
These seems like an awesome tool for system administrators. Maybe a future version could include Apache/Web Server installation and setup.

Can't wait until this comes out!

Absolutely. I've added fail2ban, mariadb, multicraft, and tcadmin. On the first release, some of these may not be available immediately. Although, they are planned.
 
Banned forever. Reason: Rules violations

OAliverpool

Currency Conversions
Supreme
Feedback score
16
Posts
474
Reactions
105
Resources
0
So you’re saying I won’t have to hire a sys admin anymore :)
 

flamelier

flamelier.com
Supreme
Feedback score
14
Posts
488
Reactions
167
Resources
0
This is awesome! It’s also nice to see that you took the suggestions from chat box as well!
 

Frank

Applications Admin III at Ferguson Enterprises
Banned
Feedback score
30
Posts
697
Reactions
402
Resources
0
So you’re saying I won’t have to hire a sys admin anymore :)

For setups, no.

The automatic scanner/diagnostic tool will fix the vast majority of common issues, but it won't fix or detect all issues. That will be a separate script by itself, but for the most part you won't have to. Especially for security, as this script will harden your server to most forms of attacks.[DOUBLEPOST=1598752884][/DOUBLEPOST]
This is awesome! It’s also nice to see that you took the suggestions from chat box as well!


Absolutely. I'm definitely open to feedback and suggestions, or ideas to add to the script. I can't promise everything, since there are some limitations, although for the most part, almost everything could be added. Feel free to leave a suggestion if you think of one. :)

I'll do my best to respond to them.[DOUBLEPOST=1598761699][/DOUBLEPOST]Update #1: Pterodactyl Installation Wizard (Partially completed)

Phase 1 installation: Preparing the environment. It will update the system, install epel and remi. (Completed)

hJieQm4Gtyeu.png


Phase 2 Installation: Installing requisites (Completed)

This will install all php extensions needed, if they are not already installed. It will also install redis, mariadb, and tar.
Speed will depend on server specs and network speed. Installs composer, makes the directory, extracts panel files, starts & enables all services needed, and completes the installation for the panel.

l61vi3Hdh24a.gif


Phase 3 Installation: Configure Database (Completed)

This part of the installation emulates mysql_secure_installation, and does a complete mysql installation without intervention needed by the user. As a result, we can create the root user with a randomly generated password and we can create another user for Pterodactyl to use with it's own details, and pretty much do anything with mysql.

After completion, you'll be presented with all the information. However, don't worry - because the information will be given to you again after everything you selected is done.

cKBn9nhxN2IX.png


All of this is done non-interactively and requires no input from the user. Later tonight I might have some other phases completed for the Pterodactyl installation. So far, i'm happy with how this has turned out.
 
Last edited:
Banned forever. Reason: Rules violations

Frank

Applications Admin III at Ferguson Enterprises
Banned
Feedback score
30
Posts
697
Reactions
402
Resources
0
Last edited:
Banned forever. Reason: Rules violations

severuscold

Feedback score
-1
Posts
35
Reactions
3
Resources
0
Looking very good so far!
Can I beta test it?
 

Frank

Applications Admin III at Ferguson Enterprises
Banned
Feedback score
30
Posts
697
Reactions
402
Resources
0
Looking very good so far!
Can I beta test it?

Maybe. I'll consider it closer towards the full release.

Is this only for CentOs currently?

It will support Debian 8, Debian 9, Debian 10 Ubuntu 18.04, Ubuntu 20.04, CentOS 7 and CentOS 8.

At first release, all these may not be supported. It depends on demand.
 
Banned forever. Reason: Rules violations

Quantum

A system admin and a developer
Supreme
Feedback score
18
Posts
56
Reactions
37
Resources
0
It's pretty cool. But I am against it because it can hurt the system admin space around here.

Edit: Well not really, but it's something
 

Anish

Developer
Supreme
Feedback score
9
Posts
365
Reactions
176
Resources
0
Did Discord Bot Maker () hurt the Discord Bot making space?
 

Frank

Applications Admin III at Ferguson Enterprises
Banned
Feedback score
30
Posts
697
Reactions
402
Resources
0
Distribution detection & compatibility detection built into script now. Version 0.5.0.

xiyLtH4XpIeH.png


3k0dmNIdapnk.png
[DOUBLEPOST=1599105299][/DOUBLEPOST]Automatic Domain detection & Interface Selection added - Version 0.5.3

UKebn0RAqkNG.png


rPRVm4AuBEib.png
 
Last edited:
Banned forever. Reason: Rules violations

SSH

The only way to reach me is discord: @ssh_
Supreme
Feedback score
30
Posts
585
Reactions
262
Resources
0
Pulling an Ethereal, nice one!
 
Status
This thread has been locked.
Top