Debugging logs can be formidable task if you run same service on multiple production loads with local logging behind a load balancer, you are only left one option, ssh into different servers and then debug the logs.
Logging to a single server from multiple servers can simply debugging, Python provides a in built functionality for logging, by just adding a few lines to the logging config you will be able to send the log to a remote server and then your remote server need to handle this request. In remote server you can store this logs into flat files or NoSQL
A rudimentary architecture would be:
I’ve created a few code samples to get this done:
Configure a HTTPHandler to the logging handler to send logs to remote server instead of local tty:
12345678910111213
importloggingimportlogging.handlerslogger=logging.getLogger('Synchronous Logging')http_handler=logging.handlers.HTTPHandler('127.0.0.1:3000','/log',method='POST',)logger.addHandler(http_handler)# Log messages:logger.warn('Hey log a warning')logger.error("Hey log a error")
On the logging server, I’ve created a simple flask application which can handle a post request:
Assuming the server is up and you send a log request, this is how the log structure looks:
123456789101112131415161718192021
{"relativeCreated":"52.1631240845","process":"10204","args":"()","module":"km","funcName":"<module>","exc_text":"None","name":"Synchronous Logging","thread":"139819818469184","created":"1446532937.04","threadName":"MainThread","msecs":"37.367105484","filename":"km.py","levelno":"40","processName":"MainProcess","pathname":"km.py","lineno":"13","msg":"Hey log a error","exc_info":"None","levelname":"ERROR"}
Important properties of this structure would be msg, name & level. Property name is what you pass to getLogger function, and level would the level of logging (error = 40, warning = 30, etc).
This approach is synchronized, if you want logging to be async use threads:
1234567891011
importlogging,thread,timeimportlogging.handlerslogger=logging.getLogger('Asynchronous Logging')# Namehttp_handler=logging.handlers.HTTPHandler('127.0.0.1:3000','/log',method='POST',)logger.addHandler(http_handler)thread.start_new_thread(logger.error,("Log error",))time.sleep(1)# Just to keep main thread alive.
By this way we need not bother about storage of application server (If you are not storing any data to FS then logs would be the only thing) and debugging would be easy.
Save to mongo to perform analytics and / or to perform quick queries:
123456789101112131415161718192021
fromflaskimportFlask,requestimportjsonfrompymongoimportMongoClientapp=Flask(__name__)# Mongo setup:client=MongoClient()db=client['logs']collection=db['testlog']@app.route('/log',methods=['POST'])defindex():# Convert form POST object into a representation suitable for mongodbdata=json.loads(json.dumps(request.form))response=collection.insert_one(data)printresponse.inserted_idreturn""if__name__=='__main__':app.run(host='0.0.0.0',port=3000,debug=True)