Writing API Documentation using RAML.

Easy to work with any API is largely dependent on how it is written and designed documentation. Ceychas we conduct work on the standardization and unification of descriptions of all our API, and documentation issues are particularly relevant for us.

After a long search we decided to issue documentation in a format RAML. So it called specialized language to describe REST API. His possibilities and advantages will be discussed in this article.

Why RAML

How should document the API? This question is not as simple as it might seem at first glance. The first and easiest option that comes to mind - provide a description of the API in the form of a plain text document. So do many people (including very well-known companies). We are not just used that way.

For all its simplicity, it has the following disadvantages:

Text documentation is difficult to keep up to date;
Often verbal API descriptions are not sufficiently descriptive;
The scope of the use of "verbal" Documentation is very limited (for example, based on it cannot generate an interactive test page).
To simplify the process of documentation, you can use specialized tools and services. As a rule, they generate documentation based on the description in a standardized format - usually JSON or Markdown.

None of these formats for writing documentation is not appropriate. JSON was originally created for exchanging data on the Web. When using it for other purposes will inevitably have to resort to "crutches" - for example, custom fields, beginning with the $ sign. In addition, make descriptions in JSON manually - it is quite routine and tedious (especially when it comes to descriptions large).
In the above-described difficulty paying attention, many users of the popular tool Swagger. Soon Swagger developers decided to simplify the job of writing specifications and created a corporate editor with support for YAML.

Of course, YAML is much more convenient than JSON. But its use is associated with certain difficulties. The fact that there are descriptions of the API always repetitive elements (e.g., the circuit response, which may be the same for different types of HTTP-requests) that have each time to register manually. Now, if they can be had once and for all to register in a separate file and refer to it in the case nebhodimo.... But, alas, is that this is not possible.

As for the format Markdown (used, for example, in the  API BluePrint), it is intended primarily for processing text, not to be used as a basis to generate. Adapt it under the API documentation is very difficult. For the same reason, have not led to any noticeable results Establish an attempt to describe the format of the API on the basis of XML - for example, language WADL (Web Application Description Language), developed by Sun Microsystems back in 2009, but has not been widely disseminated.

Founders of the project RAML (the acronym means REST-Full API Modeling Language - a language for modeling REST API) attempted to develop a language intended solely to describe the API and fix flaws inherent in other formats. The first version of the specification RAML was published in 2013. The main developer is the company RAML MuleSoft; the project also attended by representatives of such companies as Cisco, PayPal, VoxInc and others.

RAML undoubted advantages are:

Simple and logical syntax, based on the format YAML;
Support inheritance and the ability to connect external file specifications.

An additional advantage is the presence of a large number of converters, parsers and generators online documentation. Some of them are described later, but for now let's move to the Review features syntax RAML.

The System of Protection against DDoS.

The system used to protect against DDoS includes not one, but several software and hardware systems, including the Arbor Pravail and F5. Purification and analysis of the traffic carried directly into the network using specialized software tools.

This system provides protection against these types of attacks:

TCP-flood;
SYN-flood;
Illegitimate combination of TCP-flags;
Attack on TCP-session type TCP Idle, Slow TCP and others;
Attack on the HTTP-session (Slowloris, Pyloris etc.);
HTTP-flood;
DNS-flood;
DNS Cache Poisoning;
UDP-flood;
ICMP-flood;
Attack IP-, TCP and UDP-fragments;
Attacks on VoIP and SIP.

In case of attack can be used following methods:

Invalid packet List - packet filtering, do not conform to RFC;
The creation of black and white list IPv4- and IPv6-addresses;
GeoIP Filter Lists - traffic filtering by country (block traffic from countries where it comes from the highest number of DDoS-attacks).

GeoIP Policing - policed ​​traffic by country (monitoring incoming traffic and limit traffic from the countries from which comes the highest number of DDoS-attacks);
Flexible Zombie Detection - detection of zombies and creating profiles of legitimate traffic;
TCP SYN Authentication - opposition to flood through the TCP-client authentication;
DNS Authentication - combating DNS-flood through client authentication;
DNS Scoping - validation of DNS-queries with regular expressions;
DNS Malformed - checking DNS-requests to the appropriate RFC;
DNS Rate Limiting - limiting the number of DNS-queries one IP-address (suitable only for resources with a small attendance: in our country providers often use NAT. It is a typical case is when "gray" subnet / 16 comes to the Internet through a single IP and all DNS-requests go to the same address);
DNS NXDomain Rate Limiting - validation of DNS-responses. This intended for attacks in which the cache of DNS-servers overflows invalid entries; it is aimed at tracing requests from a non-existent DNS-name;

DNS Regular Expression - DNS-filtering requests on regular expressions;
TCP Connection Reset - to prevent too many TCP-connections;
Payload Regular Expression - traffic filtering by regular expression applied to Payload- packages;
HTTP Malformed - Blocking of HTTP-traffic not conforming to RFC;
HTTP Rate Limiting - limiting the number of HTTP-requests to the same IP-address;
HTTP Scoping - validation of HTTP-queries with regular expressions;
SSL Negotiation - blocking SSL-traffic not conforming to RFC;
AIF and HTTP / URL Regular Expression - the imposition of AIF signatures on analyzed traffic;
SIP Malformed - blocking SIP-traffic not conforming to RFC;
SIP Request Limiting - limiting the number of SIP-queries per IP-address.

How it works

Customers ordering the service of protection against DDoS, we provide protected IP-addresses (one address is included in the base rate, additional addresses can be ordered via the control panel). We also highlight a special lane for protected traffic. Traffic from the Internet goes to the protected address through our network of partners, where is in the process of cleaning.
All illegitimate traffic is dropped at the network partner. Customers supplied only cleared traffic. Outgoing traffic thus falls into the Internet infrastructure through us.

Launching the Service "Virtual Private Cloud".

About the service "virtual private cloud» (Virtual Private Cloud, VPC) we have already mentioned. Today we finally announce the news that many of you have been waiting for: beta-testing the new service was over, and we are launching it in a commercial operation. In this article we describe in detail about the options of using a new cloud, and that for some tariffs will be paid for its use.

Why do it: use cases

As already in a previous publication, "Virtual Private Cloud" will be of particular interest to corporate clients. With our service, they will receive the following benefits:

Significant reduction in the cost of "iron". On the basis of a rented cloud resources can be deployed hundreds of projects, and it will be much cheaper than using a real "iron": will disappear the need to purchase expensive equipment and to attract additional personnel for its service.

Easy Cost Planning. Rent cloud computing resources is tariffed according to a transparent and flexible scheme that allows you to accurately calculate the costs and forecast the economy that makes use of VPC.

The speed of the deployment infrastructure. The VPC is possible in a matter of minutes to set up a virtual infrastructure with the necessary characteristics. From properly configured cloud can, for example, without any problem to deploy IT infrastructure for the new office. With the scaling will not be serious problems arise: if necessary, you can always just buy new resources.

Let us consider in more detail the most common usage scenarios of virtual private clouds for enterprise purposes.

Building a separate infrastructure for the different departments / projects
In any organization, there is usually a number of departments or projects unrelated. If you are using a public cloud in such a case it is necessary to tie infrastructure to a single user account (which is doubtful in terms of safety), or to register a lot of accounts, which also causes a lot of inconvenience.
Virtual private cloud lets you avoid all the above problems: for each division of the organization created a separate project with limited quotas for the use of resources. People from different departments will never be able to gain access to the resources of each other.

The rank and file employees of the organization will have access only to a strictly defined virtual machines that contain strictly defined resources.
Other users with administrator rights can access the project by external control panels, in which all control functions are implemented as virtual machines and network drives, but there is no possibility of buying resources.

The transfer of the existing infrastructure to the cloud

Many organizations have to deal with a situation where support its own virtualized infrastructure starts too costly: requires the organization of a redundant power supply and connect to the Internet, attracting additional staff, purchase of equipment and replacement of faulty components... It is logical solution in such a situation, it is the transfer of the entire infrastructure or a part the cloud. But it is often difficult: It is necessary to reinstall the software from scratch. An additional complication is that cloud providers often impose a lot of restrictions, because of what is necessary to make significant changes to the architecture.

Especially for corporate customers who want to migrate their own infrastructure to us, we took a number of measures to facilitate the migration:

In our cloud allows you to import ready-made virtual machine (it supports all common formats);
Network topology in our VPC can be configured in an arbitrary manner; If you and the public network and use the local network (or even a few local area networks), all this can be easily implemented in our country;

We use virtualization allows you to run the virtual machines running any operating system - if you ever need, for example, Solaris, or FreeBSD, they can be set without problems.

DDoS Attacks – A brief Overview.

We provide a new service for protection from DDoS-attacks. It is designed for users of "dedicated server" and "Placing a server," as well as for customers who rent server rack.

To learn how to be protected, we describe in detail in this article.

DDoS Attacks: A brief overview
Abbreviation is DDoS Distributed Denial of Service - attack distributed denial of service. DDoS-attacks called malfunction attacked machines by sending requests to it from multiple hosts.

Typically, DDoS-attacks are carried out through a botnet - a network of computers that have installed malicious software (called zombification).

Some types of attacks can be carried out without a botnet (e.g., UDP-flood).

By all objective DDoS Attacks can be divided into the following groups:

The attacks that aim to overload the bandwidth. Examples of this type of attack can serve as mentioned above UDP-flood, ICMP-flood (aka ping flood), and other practices of sending packets that are not requested. The strength of such an attack is measured in gigabits per second. It is constantly growing and now can be up to 100 or more gigabits per second.

Attacks on the protocol level. As the name implies, this type of attack using the vulnerability and limitations of various network protocols. They "bombard" the server parasitic package and it becomes unable to process legitimate user requests. As an example, SYN-flood, teardrop and other attacks that disrupt the normal movement inside a package at various stages of the protocol.

The attacks on the application layer disrupt the normal functioning of the system using a vulnerability and weaknesses of applications and operating systems.

For more information on the issues of classification of DDoS-attacks, we shall not stop - the interested reader can easily find the Internet a variety of materials on the subject. Much more interesting to us are the existing methods of protection against DDoS. They will be discussed below.

Methods of protection against DDoS

Methods of protection from DDoS-attacks are divided into two large groups: methods of prevention and response methods.

To prevent DDoS-attacks are commonly used hardware methods of perimeter security - firewall combined with intrusion detection system (Intrusion Detection Systems, IDS). However, protection in a strict sense they do not provide.

DDoS-attack is quite possible to organize and within the permitted firewall packages. Regarding IDS, they typically operate within the signature and the statistical analysis, comparing the incoming packets with the existing traffic patterns. If the attack is carried out by conventional distribution network packets that individually are not malicious, not all IDS can detect it.

In addition, and firewalls, and IDS devices are often controlled by the sessions, so can themselves become the object of attack.

An effective means of minimizing downtime when DDoS-attacks are multiple reservation - Organization of clusters of servers placed in different data centers and connected to different communication channels. If one of the components of such a system will fail, clients will be redirected to the server is running. This method has only one drawback: the construction of a distributed cluster with multiple redundancy requires a very large financial costs.

The methods used to respond in situations where an attack has already begun and it must be stopped (or at least minimize its consequences).

If the purpose of the attack is a single machine, you can simply change its IP-address. The new address is then possible to give only the most reliable external users. This solution is hardly ideal, but it is quite effective.

In some cases, help filtering techniques. After analyzing malicious traffic can be found in him a certain signature. Based on the results of the analysis can build a router ACL or firewall rules.
In addition, most of the attacking traffic often comes from a particular provider or backbone router. In this situation, a possible solution is to lock the direction from which traffic enters a questionable (it should, however, bear in mind that legitimate traffic in this case will also be blocked).

Console Version of Cyberduck - Working with Cloud Storage.

Load Balancing - Basic Algorithms and Methods

The issue of capacity planning should be addressed more in the early stages of any web project. "The Fall," the server (as it always happens unexpectedly, at the wrong time) is fraught with very serious consequences - both morally and financially. Initially, the problem of lack of server performance due to increasing loads can be solved by increasing server capacity, or to optimize the algorithms, code, and so on. But sooner or later there comes a time when these measures are insufficient.

We have to resort to clustering: multiple servers together in a cluster; the load is distributed between them using a set of special techniques called balancing. In addition to addressing the problem of high loads clustering helps to ensure redundancy of servers to each other.

The effectiveness of clustering depends on how distributed (balanced) load across cluster members.
Load balancing can be carried out using both hardware and software tools. On the basic methods and algorithms and balance we would like to tell you in this article.

Levels Balancing:
Balancing procedure is performed using a whole range of algorithms and techniques that comply with the following levels of model OSI:
Network;
Transport;
Applied.
Consider these levels of detail.

Algorithms and methods for balancing
There are many different algorithms and methods for load balancing. When choosing a particular algorithm, we must proceed in the first place, the specifics of a particular project, and secondly - of the goals. Which we plan to achieve.

Among the purposes for which use balancing, you need the following:
Justice: it is necessary to ensure that the processing of each request allocated system resources and prevent the occurrence of situations where a request is processed, and all the others waiting for their turn;

Efficiency: all of the servers that process the requests, must be occupied by 100%; it is desirable to avoid a situation where one server is idle in anticipation of requests for treatment (immediately specify that in actual practice, this goal is achieved not always);

Reducing the time of the query: need to ensure a minimum time between the beginning of a query (or queued for processing) and its completion;

Reducing the response time: it is necessary to minimize the response time to user requests.

It is also desirable that the balancing algorithm had the following properties:
Predictable: it is necessary to clearly understand in what situations and under what load algorithm will be effective for the task;
Uniform loading of system resources;
The algorithm should continue to operate when the load increases.

Colocation: Simple, Accessible, Reliable.

The service "Location Server", sometimes also called "colocation" - one of the most popular in our company. From the beginning, we tried to make it as close as possible to the scheme "set and forget". The client is only required to ensure the delivery server in the data center, and everything else (Rack, an Internet connection, and so on), we completely take over. Also, anyone who puts our equipment, we offer a wide range of additional services.

In this article we would like to share their own experiences on the organization of services "Placing the server." The whole procedure of interaction with the client is subdivided into a few simple steps, which will be discussed below.

Step 1: purchase and delivery of the server in the data center
By purchasing equipment from any supplier, you can immediately order its delivery in one of our data centers (an employee of the shipping service, which will bring the server must first be formalized power of attorney for Handover). The personal presence of the customer when receiving the equipment to colocation with us is entirely optional. All the necessary documents then we will send by registered mail.
Through this approach, placing their equipment may have customers from other cities and even countries.

Step 2: Installation
All client equipment is located in our own data center level TIER III. The detailed specifications of data centers can be found on our website.

The server rack you do not have to install their own: the installation of equipment is carried out exclusively by our own.

For security reasons, access to the server racks have only our engineers; Customer access to colo-resistant prohibited.

For customers placing much power equipment, there are special services: allocation of additional power outlet and extra electrical outlets to connect redundant power supplies servers.

Step 3: Connect to the Internet and providers
All network infrastructure we undertake; you may only need to order the necessary services in the client panel.

The service "Location Server" is a guaranteed channel 100 Mbit / s, completely unlimited, without any constraints and relationships. On request Internet connection speed can be increased up to 10 Gbit / s (with the appropriate network card). For customers with large volumes of traffic are attractive package deals.

You can always order extra IPv4 and IPv6-addresses.
If you have a stand-alone system with its own dedicated IP-address block, we are ready to provide her with the announcement of the Internet through our equipment. You can also set us BGP-session and announce their own prefixes.

If you want to connect placed our equipment to existing resources (e.g., corporate networks or traffic exchange points), you can order cross-connect with operators in the data center.

Servers posted and rented from us can be united in a local network. It concerns including servers located in different data centers, and even in different cities.

For customers who demand higher security and data protection, the possibility of renting a hardware firewall. Service is also provided protection against DDOS-attacks.

Step 4: Configure Server
After installing the equipment it needs to adjust. To do this, you do not need to come to the data center: your server, you can manage remotely.

Upon request by the ticket system available free of charge temporary connection of IP-KVM (permanent connection is possible for an additional fee).

If the server supports IPMI, iLO, or their equivalents from other manufacturers, you can rent an additional dedicated port on the network equipment to gain access to your server from outside.

Step 5: Maintenance and Support
So, the server is installed in a rack, connected to the Internet, is set. All further work on its maintenance, we take over.

If you need to restart the server - just send a request via the ticket system.
The work is not stipulated in the contract (additional configuration, software installation, diagnostics and troubleshooting, etc.) can be performed in within the service pay administration.
For a fee, we can organize the storage of spare parts for your equipment on their territory. In case of problems we are ready to carry out repairs and replace parts on their own (in this case the conditions are discussed individually).

Conclusion
Recently, we had a lot of work on the development and improvement of services "Placing the server." Today, it corresponds to the principle of "set and forget", and that have been able to make hundreds of our clients.
If you have ideas for improving services - welcome to comment.