3  Introduction

The YumaPro Tools suite provides automated support for development and usage of network management information.

All management data is defined with the YANG data modeling language.

All management operations are encoded in XML 1.0 and performed with standard NETCONF protocol operations.

3.1  System Components

Image1

The following external program is used by YumaPro, and needs to be pre-installed:

The following external libraries are used by YumaPro, and need to be pre-installed. They are usually installed by default and do not need to be installed by you:

The following external library is built within YumaPro and does not need to be pre-installed:

The following shared (or static) library is built by YumaPro and used by almost all of its programs:

 

The following libraries are built by YumaPro, and used within executables:

 

The following binaries are built by YumaPro:

The following sample netconfd-pro module instrumentation library is provided as an example.  These libraries can only be created with the YumaPro SDK.  Refer to the YumaPro Developer's Guide for details on creating server instrumentation libraries.

3.1.1  YANG

A YANG module define the semantics and syntax of a specific management feature. They are similar to SMIv2 (MIB) modules, but much more powerful and extensible.  YANG provides the ability to define a detailed programmatic interface utilizing all protocol features:

Network management software developers creating a new management feature start by defining the YANG module(s) for the NETCONF representation of the feature.  This can include any mixture of new operations, data, and notifications.  Existing YANG modules can be augmented as well.

YANG provides complex nested data structures and choices, which allows data modelers to design management interfaces which closely resemble the native data structures within the server implementation.

It is easy to get started with YANG, and there are many optional advanced features that can be utilized as well.  YANG provides many machine-readable constructs which allow YumaPro to automate many aspects of network management software development.

 

 

graphics6

Semantics and details that are usually only found in 'description' clauses can be understood and implemented automatically by the software tools.

A YANG module can be a single file, or it can be split into an arbitrary number of files, using sub-modules.  A YANG submodule is essentially the same as a main module, except that the namespace URI value is shared between the main module and all its submodules.

A submodule is referenced with the include statement instead of the import statement.

Submodules can also include other submodules, except a loop may not be formed by the include statements.

Conceptually, the module is not nested.  All definitions in submodules appear at the top level of the YANG module, even submodules included by other submodules.

All YANG modules and submodules have revision dates.  The example shows a simple version number, but the actual revision strings are date strings in the form 'YYYY-MM-DD'.

YumaPro programs support concurrent usage of different revisions of the same module or submodule.  This can occur via groupings from external modules within the YANG language.  Only one revision of a module can be imported into a single module or submodule, but any of these files may in turn import other modules.  It is possible that a different version of the same module could be indirectly imported in this case.

Deviation modules are normal YANG modules, except they only contain deviation statements.  These deviation statements are used to alter (patch) the YANG modules with implementation-specific differences.

A deviation module can contain any number of deviation statements, and they can apply to an arbitrary number of objects, from any module.  Multiple deviation statements for the same target will be combined by the server before using them, and all deviate statements for the same object will be validated together, as if they were all contained in the same deviation statement.  The order of the deviation statements is irrelevant.

Deviations modules are processed first, and the deviation statements save for later.  The import statements are ignored, unlike real module processing.

Since deviation modules are not identified in any way, YumaPro programs use the --module parameter to refer to a normal YANG module or submodule, and the --deviation parameter to refer to a deviation module.

 

graphics7

The YANG feature statement is used to define a conceptual partition within the module.

Objects that contain the if-feature statement for the corresponding feature are part of the feature.

If the server does not advertise a feature in its <capabilities>, then it is not supported, and all the objects that are part of the feature are not supported.

Multiple if-feature statements form a logical AND expression.  All the referenced features must be enabled for the object to be available.  In the example above, leaf 'YY2' is not present unless feature A and B are both advertised by the server.

3.1.2  NETCONF

The mandatory components of the NETCONF protocol are defined in RFC 6241 and RFC 6242.

 

graphics8

 

The NETCONF protocol is used to provide secure access all YANG content.  The server maintains a database which is accessed as if it was an XML instance document.

 

graphics1

Data can be retrieved with XML (subtree) or XPath filters.  Changes can be validated before being activated. Databases can be locked to prevent multiple managers from interfering with each other.  Custom operations can be used to perform complex actions and perhaps return some data as well.

 

graphics10

NETCONF can utilize several secure transport protocols.  The mandatory transport (SSH2) is used by YumaPro.  The OpenSSH server is used in the netconfd-pro implementation, and libssh2 library is used in the yangcli-pro implementation, to provide all SSH2 layer support.

By default, TCP port 830 (netconf-over-ssh) is used for all NETCONF communications between yangcli-pro and netconfd-pro.   TCP port 22 (ssh) is also supported by default, and additional TCP ports can be configured.

NETCONF security is session-based.  Privileges are granted to a session based on the username provided in the SSH connection setup.

Access control is configurable (via ietf-nacm.yang or yuma-nacm.yang), based on group membership.  The access control rules permit or deny access to one or more groups, to a subset of the YANG content.  Separate defaults for read, write, and exec (RPC operation) access are provided.

3.1.3  YANG-based Automation

YumaPro is a 100% “native YANG” implementation.  This means that YANG modules are used directly by all the tools to control all aspects of NETCONF protocol usage.  There are no lossy translations, or complicated configuration steps, in order to use a YANG module.  Simply load a module and start using it.

The automation concepts will be familiar to SNMP developers who use SMIv2 to write MIB modules.  The SMIv2 language contains enough machine-readable clauses so a client and server can automate certain aspects of the SNMP protocol implementation.

YANG does the same thing for NETCONF developers, only 10 times better.

 

graphics9

There are many more machine-readable constructs in YANG, and more powerful data modeling features.  The complicated YANG features are optional, so traditional 'DESCRIPTION clause' based semantics are still supported.

The more machine-readable YANG clauses that are used, the more the yangcli-pro client and netconfd-pro server can automate the entire NETCONF protocol implementation.

The YANG language includes many ways to specify conditions for database validity, which traditionally are only documented in DESCRIPTION clauses:

YANG Automation Constructs

 

YANG statement

description

config boolean;

The config statement indicates if the object is writable, or read-only.  The server uses this information when automatically skipping config=false entries for the <get-config> operation.

default string;

The default statement specifies the mandatory-to-use default value, if no leaf is provided.  Unlike SMIv2 DEFVAL, it is not a suggestion, and the client can rely on it.  Defaults can be specified in typedef or leaf statements.  If both are defined, then the leaf default will be used.

YANG statement

description

deviation deviation-target-path { … }

The deviation statement allows any YANG object be customized for a particular platform or implementation..  The tools can automatically support the altered objects,  based on the sub-statements within the deviation statement.  These changes can be of any nature, even those normally not allowed in YANG.  The intent of the deviation statement os to accurately describe the object implementation, so the tools can automate the protocol operations correctly, even for non-standard implementations.

error-app-tag apptag-string;

The error-app-tag statement can be used within the range, length, and pattern statements.  If a value is invalid due to the corresponding error, then the <error-app-tag> field in the <rpc-error> sent by the server will be set to the 'apptag-string'  value.

error-message errmsg-string;

The error-message statement can be used within the range, length, and pattern statements.  If a value is invalid due to the corresponding error, then the <error-message> field in the <rpc-error> sent by the server will be set to the 'errmsg-string' value.

extension

The extension statement allows a vendor to add language extensions, and all YANG implementations must be able to parse the extension correctly.  However, only implementations which actually understand the extension will support it.  All others will simply ignore the extension.

feature

The feature statement allows a module to be conceptually partitioned into mandatory and conditional object groups.  All objects with the corresponding if-feature statement will be present only if the feature is supported by the server.

if-feature feature-name;

Construct containing the if-feature statement is only included if the specified feature is supported by the server.  Otherwise, the object does not exist on the server.

import (by revision)

The import statement allows definitions from other modules to be used.  A specific revision date can be used within the entire module.  However, it is possible that different versions of imported typedefs and groupings can be used, if one imported module also imports some modules.

include (by revision)

The include statement provides the exact same features as the import statement, except it applied to sub-modules included within a module (or other sub-modules), instead of other modules.  It allows multiple sub-modules to be combined to create one conceptual YANG module.

key key-leaf-list;

The key statement indicates a set of one or more top-level leafs within the list that are used to name a specific instance of the particular list object.  All protocol operations, such as <edit-config>, can be fully automated, based on the information in this statement.

length length-spec-string;

The length statement is exactly like the range statement, except it limits the length of string leaf and leaf-list objects.

YANG statement

description

mandatory boolean;

The mandatory statement indicates that the choice, list or leaf must be provided by the client.  It will not be created by the server.  Most parameters are not mandatory however, so the default is 'false' if this statement is missing.

max-elements
 number | 'unbounded' ;

Specifies the maximum number of instances that a list or leaf-list object can have in a valid database. The default is 'unbounded', if this statement is not present.

min-elements number;

Specifies the minimum number of instances that a list or leaf-list object must have in a valid database. The default is zero, if this statement is not present.

must xpath-expr;

If the object containing the must statement exists, then the XPath expression must evaluate to 'true' for the database to be valid.  This provides referential integrity checks among related parameters.

pattern pattern-string ;

The pattern statement specifies a regular expression that must evaluate to 'true' in order for the corresponding string leaf or leaf-list object to be valid.  Multiple patterns encountered in a nested typedef chain must all evaluate to 'true' for the object to be valid.

range range-spec-string ;

The type statement can specify the range of a numeric type.  Since typedefs can be nested in YANG, the range statements are nested also, and constitute an AND expression (i.e., all the range tests must pass in the chain of type definitions.)  THe keywords 'min' and 'max' indicate the minimum and maximum values from the parent typedef (if any), not the built-in type.

refine refine-target-path { … }

The refine statement is defined  within a uses statement, and allows the specific grouping to be customized for each individual copy of the grouping contents.  The tools can automatically support the refined objects, based on the sub-statements within the refine statement.

revision revision-date { … }

The revision statement identifies the most current version of a YANG module or sub-module.  Multiple versions at once are supported in YANG.

unique unique-node-list;

The unique statement indicates an arbitrary tuple of descendant nodes within a list, which have to be unique within the list.  These nodes are not keys, and can be nested anywhere within a single list entry.

uses grouping-name;

The uses statement inserts an instance of a reusable grouping, replacing the uses node within the conceptual data tree.

when xpath-expr;

The object containing the when statement is only allowed to exist if the XPath expression evaluates to 'true'.  This provides  a SPARSE AUGMENTS capability when combined with the augment statement.

 

 

3.1.4  YANG Language Extensions

There are several YANG extensions that are supported by YumaPro.  They are all defined in the YANG file named yuma-ncx.yang.  They are used to 'tag' YANG definitions for some sort of automatic processing by YumaPro programs.  Extensions are position-sensitive, and if not used in the proper context, they will be ignored.  A YANG extension statement must be defined (somewhere) for every extension used in a YANG file, or an error will be occur.

Most of these extensions apply to netconfd-pro server behavior, but not all of them.  For example, the ncx:hidden extension will prevent yangcli-pro from displaying help for an object containing this extension.  Also, yangdump-pro will skip this object in HTML output mode.

The following table describes the supported YANG language extensions.  All other YANG extension statements will be ignored by YumaPro, if encountered in a YANG file:

 

YANG Language Extensions

 

extension

description

ncx:hidden;

Declares that the object definition should be hidden from all automatic documentation generation.  Help will not be available for the object in yangcli-pro.

ncx:metadata
  “
attr-type
  attr-name
”;

Defines a qualified XML attribute in the module namespace. Allowed within an RPC input parameter.

attr-type is a valid type name with optional YANG prefix.

attr-name is the name of the XML attribute.

ncx:no-duplicates;

Declares that the ncx:xsdlist data type is not allowed to contain duplicate values.  The default is to allow duplicate token strings within an ncx:xsdlist value.

ncx:password;

Declares that a string data type is really a password, and will not be displayed or matched by any filter.

ncx:qname;

Declares that a string data type is really an XML qualified name. XML prefixes will be properly generated by yangcli-pro and netconfd-pro.

ncx:root;

Declares that the container parameter is really a NETCONF database root, like <config> in the <edit-config> operations.  The child nodes of this container are not specified in the YANG file.  Instead, they are allowed to contain any top-level object from any YANG file supported by the server.

ncx:schema-instance;

Declares that a string data type is really an special schema instance identifier string.  It is the same as an instance-identifier built-in type except the key leaf predicates are optional.  For example, missing key values indicate wild cards that will match all values in nacm <data-rule> expressions.

nacm:secure;

Declares that the database object is a secure object.

If the object is an rpc statement, then only the netconfd-pro 'superuser' will be allowed to invoke this operation by default.

Otherwise, only read access will be allowed to this object by default,  Write access will only be allowed by the 'superuser', by default.

extension

description

ncx:user-write <bits>;

Declares the user write permissions that will be allowed for a config=true data node.  The bits parameter contains the write permissions that will be allowed for the object (create, update, delete).

nacm:very-secure;

Declares that the database object is a very secure object.

Only the 'superuser' will be allowed to access the object, by default.

ncx:xsdlistlist-type;

Declares that a string data type is really an XSD style list.

list-type is a valid type name with optional YANG prefix.

List processing within <edit-config> will be automatically handled by netconfd-pro.

ncx:xpath;

Declares that a string data type is really an XPath expression. XML prefixes and all XPath processing will be done automatically by yangcli-pro and netconfd-pro.

 

3.1.5  YANG Compiler

The YumaPro programs all use the same centralized YANG language parser.

The complete YANG language is supported, as defined in RFC 6020. The file naming conventions defined in this specification must be used, along with all the language definition rules.

Definitions can be contained in modules and/or sub-modules.

Any number of revisions of a module or submodule can be used concurrently,  The import-by-revision and include-by-revision features of YANG are fully supported,  Refer to the section 'Searching for Files' for more details.

All extension usage within YANG files is supported and saved.  The application data is available to all YumaPro programs, including netconfd-pro server instrumentation.  Refer to the 'YANG User Guide' for details on writing YANG files and using the extensions built into YumaPro.

Note: The smidump is not part of YumaPro, but it can be utilized to convert MIB modules written in SMIv2 into YANG modules, which can then be implemented in netconfd-pro, and managed with yangcli-pro.  The freely available libsmi library contains the smidump program.

3.1.6  YANG Module Library

The central system component is the set of YANG data model modules which define all available management information.  This set of modules is expected to grow over time, and there is usually a high degree of reuse and inter-dependence between the modules.

YANG modules can import other modules to reuse any definitions, and to augment objects in other modules.  Each module represents one unique XML namespace used within the NETCONF protocol.  A module can be partitioned into any number of submodules, each in a separate YANG file.  The submodules are conceptually combined, and only the entire module is accessible to other modules.

 

Directory Layout

 

YumaPro can utilize several directories to store files used during operation.  By default, a 'root' directory and all of its sub-directories are searched for these files. Several different roots can be searched.  Generally, there is one centralized root (YUMAPRO_INSTALL) shared by all users, and one or more 'project' roots (YUMAPRO_HOME), which can be shared but may belong to a single user.

The YumaPro programs need to find and store the following types of files during operations:

The search paths used to find these files are discussed in detail in the System Configuration section.

 

graphics4

 

Module Revisions

 YANG has extensive module life cycle support.  Each module or submodule has a revision date, and multiple revisions of the same module or submodule may be used at once within the same server.

The YANG module repository is the authoritative source of common management information for the netconfd-pro server.  However, different platform implementations of the same data model need to be 'adjusted' slightly to reflect differences in the feature  support available on each platform.

YumaPro has an extensive set of mechanisms to automate the maintenance of these platform-specific 'special requirements'. A single YANG module (plus 'patches' and deviations as needed for each platform) can be published, instead of a separate version of the YANG module for each platform.

graphics3

 

 

Module Naming Conventions

 

YANG module names are usually lower-case. Hyphen (-), underscore (_) and period (.) characters are allowed, after the first character, which must be a letter.  It is suggested that only the at sign (@) character be used as a separator between module name string components.  YANG files must use the suffix '.yang'.  YIN files must use the suffix 'yin'.

There are two forms of YANG file names: with and without a revision date.

 

     module.yang

ietf-netconf-monitoring.yang      (no revision or unspecified revision)

 

     module@revision-date.yang

ietf-netconf-monitoring@2009-04-17.yang   (must be the 2009-04-17 version)

 

These naming conventions are important when YumaPro needs to resolve an 'import' or 'include' statement in a YANG file.  Refer to section 4.2 for more details on YANG module search paths and the 'import-by-revision' feature of YANG.

3.1.7  YANG Files

YANG modules and submodules are text files encoded in UTF-8. . There is also an alternate XML encoding called YIN.  Sometimes the term YANG module is used to refer to the conceptual module, whether it is encoded in YANG format or YIN format.

All YumaPro Tools programs will accept either encoding format, however line and column numbers are not correct in log messages for YIN encoded modules.  Instead, each XML node is given a monotonically increasing value, and the XML document order is used instead of line numbers in error/warning messages for YIN files.  The column number is always '1' for YIN files.

A module can be validated and checked for possible programming mistakes, by using the yangdump-pro program.  Many 'reports' can also be generated:

The yangdump-pro program is also used to generate other files, derived from the YANG content:

3.1.8  NETCONF Managers

The NETCONF client is an application that initiates and utilizes NETCONF sessions to control and monitor a NETCONF server.

YumaPro includes the yangcli-pro application for this purpose.  It can be used as a stand-alone tool with any NETCONF server.

3.1.9  NETCONF Servers

The NETCONF server is a server application that is always running on the managed device.  It listens for NETCONF session requests from a NETCONF client, and allows specific users to access specific subsets of the available content (operations, database access, and notifications).  It processes all incoming protocol operation requests from the client, and insulates all the instrumentation code from these protocol operations.

YumaPro includes the netconfd-pro application for this purpose.  It can be run on several different platforms, or easily adapted to embedded platforms.