skyline: Skyline 是实时异常检测系统,构建目的是为了实现对成千上万的指标进行被动监控,不需要一个个配置模型/阈值,可以同时使用 Nagios | 您所在的位置:网站首页 › skyline实时 › skyline: Skyline 是实时异常检测系统,构建目的是为了实现对成千上万的指标进行被动监控,不需要一个个配置模型/阈值,可以同时使用 Nagios |
Skyline is an Archived Project
Skyline is no longer actively maintained. Your mileage with patches may vary. SkylineSkyline is a real-time* anomaly detection* system*, built to enable passive monitoring of hundreds of thousands of metrics, without the need to configure a model/thresholds for each one, as you might do with Nagios. It is designed to be used wherever there are a large quantity of high-resolution timeseries which need constant monitoring. Once a metrics stream is set up (from StatsD or Graphite or other source), additional metrics are automatically added to Skyline for analysis. Skyline's easily extendible algorithms automatically detect what it means for each metric to be anomalous. After Skyline detects an anomalous metric, it surfaces the entire timeseries to the webapp, where the anomaly can be viewed and acted upon. Read the details in the wiki. Installsudo pip install -r requirements.txt for the easy bits Install numpy, scipy, pandas, patsy, statsmodels, msgpack_python in that order. You may have trouble with SciPy. If you're on a Mac, try: sudo port install gcc48 sudo ln -s /opt/local/bin/gfortran-mp-4.8 /opt/local/bin/gfortran sudo pip install scipyOn Debian, apt-get works well for Numpy and SciPy. On Centos, yum should do the trick. If not, hit the Googles, yo. cp src/settings.py.example src/settings.py Add directories: sudo mkdir /var/log/skyline sudo mkdir /var/run/skyline sudo mkdir /var/log/redis sudo mkdir /var/dump/Download and install the latest Redis release Start 'er up cd skyline/bin sudo redis-server redis.conf sudo ./horizon.d start sudo ./analyzer.d start sudo ./webapp.d startBy default, the webapp is served on port 1500. Check the log files to ensure things are running.Debian + Vagrant specific, if you prefer GotchasIf you already have a Redis instance running, it's recommended to kill it and restart using the configuration settings provided in bin/redis.conf Be sure to create the log directories. Hey! Nothing's happening!Of course not. You've got no data! For a quick and easy test of what you've got, run this: cd utils python seed_data.pyThis will ensure that the Horizon service is properly set up and can receive data. For real data, you have some options - see wiki Once you get real data flowing through your system, the Analyzer will be able start analyzing for anomalies! AlertsSkyline can alert you! In your settings.py, add any alerts you want to the ALERTS list, according to the schema (metric keyword, strategy, expiration seconds) where strategy is one of smtp, hipchat, or pagerduty. You can also add your own alerting strategies. For every anomalous metric, Skyline will search for the given keyword and trigger the corresponding alert(s). To prevent alert fatigue, Skyline will only alert once every for any given metric/strategy combination. To enable Hipchat integration, uncomment the python-simple-hipchat line in the requirements.txt file. How do you actually detect anomalies?An ensemble of algorithms vote. Majority rules. Batteries kind of included. See wiki ArchitectureSee the rest of the wiki Contributions Clone your fork Hack away If you are adding new functionality, document it in the README or wiki If necessary, rebase your commits into logical chunks, without errors Verfiy your code by running the test suite and pep8, adding additional tests if able. Push the branch up to GitHub Send a pull request to the etsy/skyline project.We actively welcome contributions. If you don't know where to start, try checking out the issue list and fixing up the place. Or, you can add an algorithm - a goal of this project is to have a very robust set of algorithms to choose from. Also, feel free to join the skyline-dev mailing list for support and discussions of new features. (*depending on your data throughput, *you might need to write your own algorithms to handle your exact data, *it runs on one box) |
CopyRight 2018-2019 实验室设备网 版权所有 |