10  Development Environment

This section describes the YumaPro Tools development environment used to produce the Linux binaries.

 

10.1  Programs and Libraries Needed

There are several components used in the YumaPro software development environment:

 

The following external program is used by YumaPro, and needs to be pre-installed:

 

The following program is part of YumaPro Tools, and needs to be installed:

 

The following program is part of YumaPro Tools, and usually found within the YumaPro development tree:

 

The following external libraries are used by YumaPro, and need to be pre-installed:

 

10.2  SIL Shared Libraries

The SIL callback functions are built as shared libraries that are loaded dynamically and accessed by the server with the dlopen and dlsym library functions.  The user callback functions and their source files use the “u_” prefix. YumaPro callback functions and their files use the “y_” prefix.

SIL callback code is executed in the netconfd-pro process.

The following YANG statements are supported in SIL code:

The following YANG statement is supported in SIL code if built into a SIL bundle

 

graphics24

The make_sil_dir_pro script can be used to generate the C and H file code stubs for a single YANG module. This script uses the yangdump-sdk program to generate these files.  In this mode, SIL code will be generated only for the definitions in the target module. External augment statements from the target module (or any other module) will not affect SIL code generation.  

SIL libraries for single modules are loaded into the server with the --module parameter.

The make_sil_bundle script can be used to generate the C and H file code stubs for a set of YANG modules. This script uses the yangdump-sdk program to generate these files, but with the --sil-bundle parameter set..  In this mode, SIL code will be generated for the definitions in the target module. External augment statements from all the modules loaded together will affect SIL code generation.

SIL libraries for SIL bundles are loaded into the server with the --bundle parameter.

SIL libraries cannot be generated for YANG submodules. When using yangdump-sdk to generate SIL files, use the --unified=true option to generate SIL code for the entire module.

 

10.2.1  SIL Library Names

SIL libraries must be named with a predictable string in order for the server to find it with minimal configuration.

The name must be “lib” + <module-name> + “.” + <file-ext>

where:

 

Example:


test.yang → test.c → libtest.so

 

10.2.2  SIL Library Location

By default, SIL binaries are installed in the /usr/lib/yumapro directory (/usr/lib64/yumapro on Fedora x64 platforms).

The YUMAPRO_RUNPATH (or --runpath CLI parameter) is used to specify alternate locations for SIL library binary files.

 

10.2.3  make_sil_dir_pro

The make_sil_dir_pro script should be used to generate SIL code stub files for a single YANG module.

The --split parameter should also be used to generate separate user and system code stubs.  This is the recommended method in order to minimize coding effort.  Combined SIL code stubs (without the --split option) can be useful for highly optimized or customized callback design.

The script will generate a directory sub-tree and fill in the Makefiles and source files.

Example:

 

> make_sil_dir_pro --split address

 

 

graphics13

Refer to the make_sil_dir_pro manual page (“man make_sil_dir_pro”) for more details.

 

10.2.4  make_sil_bundle

The make_sil_bundle script should be used to generate SIL code stub files for multiple YANG modules.

The parameters are a list of strings.

The first parameter is the bundle name. Any additional parameters are module names. These module names should be listed so the base modules are specified first and modules that augment the base modules are listed last.

The script will generate a directory sub-tree and fill in the Makefiles and source files.

Example:

 

 

> make_sil_bundle ifbundle ietf-interfaces ietf-ip

 

 

In this example, a bundle named 'ifbundle' is created. The same directory structure is used as for a single module.  C files are created for the bundle as well as the module files.  The command --bundle=<bundle-name> is used to load a SIL bundle into the server at boot-time.

 

10.2.5  Building SIL Libraries

The compiler flags passed to “make” are extremely important when building SIL source code. The flags must align with those used to build the main server.

Specifically, the “yumapro-pthreads” server must use SIL code that is compiled with the PTHREADS=1 compiler flag. This flag is used in some H files and therefore data structures will be misaligned if the same value is not used in both the server and the SIL libraries.

The version string for a PTHREADS server build contains the phrase “THD” at the end. The command “netconfd-pro --version” can be used to print the server version string.

The DEBUG=1 flag can also be used when building SIL code. This will cause the --ggdb compiler flag to be used.

 

10.3  SIL-SA Libraries

The SIL-SA callback functions are usually built as shared libraries that are loaded dynamically and accessed by an external application process with the dlopen and dlsym library functions.  

A SIL-SA library can also be built as a static library and linked to the “sil-sa-app” or “combo-app” programs.

The user callback functions and their source files use the “u_” prefix. YumaPro callback functions and their files use the “y_” prefix.

SIL-SA callback code is executed in your external process.

The following YANG statements are supported in SIL-SA code:

The following YANG statements are NOT supported in SIL-SA code at this time. This includes:

 

graphics27

The make_sil_sa_dir script can be used to generate the C and H file code stubs for a single YANG module. This script uses the yangdump-sdk program to generate these files.  In this mode, SIL code will be generated only for the definitions in the target module. External augment statements from the target module (or any other module) will not affect SIL code generation.  

SIL libraries for single modules are loaded into the server with the --module parameter.

The make_sil_sa_bundle script can be used to generate the C and H file code stubs for a set of YANG modules. This script uses the yangdump-sdk program to generate these files, but with the --sil-bundle parameter set..  In this mode, SIL code will be generated for the definitions in the target module. External augment statements from all the modules loaded together will affect SIL code generation.

SIL-SA libraries for SIL-SA bundles are loaded into the proper subsystem with the --bundle parameter on the server.

SIL-SA libraries cannot be generated for YANG submodules. When using yangdump-sdk to generate SIL files, use the --unified=true option to generate SIL code for the entire module.

 

10.3.1  SIL-SA Library Names

SIL libraries must be named with a predictable string in order for the server to find it with minimal configuration.

The name must be “lib” + <module-name> + “_sa” + “.” + <file-ext>

where:

 

Example:


test.yang → test.c → libtest_sa.so

 

 

10.3.2  SIL-SA Library Location

By default, SIL-SA binaries are installed in the /usr/lib/yumapro directory (/usr/lib64/yumapro on Fedora x64 platforms).

The YUMAPRO_RUNPATH (or --runpath CLI parameter) is used to specify alternate locations for SIL library binary files.

 

 

10.3.3  make_sil_sa_dir

The make_sil_sa_dir script should be used to generate SIL-SA code stub files for a single YANG module.

The --split parameter should also be used to generate separate user and system code stubs.  This is the recommended method in order to minimize coding effort.

The script will generate a directory sub-tree and fill in the Makefiles and source files.

Example:

 

 

> make_sil_sa_dir --split address

 

graphics26

Refer to the make_sil_sa_dir manual page (“man make_sil_sa_dir”) for more details.

10.4  SIL-SA Bundles

SIL-SA bundles are similar to a SIL-SA for a single YANG module, but a bundle can contain multiple YANG modules.  This is useful to simplify development and/or support automatic code stub generation for augment statements that add definitions to another YANG module.

The same YANG usage restrictions for SIL bundles apply to SIL-SA bundles.

10.4.1  make_sil_sa_bundle

The make_sil_sa_bundle script should be used to create SIL-SA code stubs for multiple YANG modules. Just one SIL-SA library file will be produced for the entire bundle.

 

graphics25

 

 

 

 

10.5   Static SIL-SA Libraries

The SIL-SA library for a module or a bundle can be built as a static library and linked directly into the “sil-sa-app” or “combo-app” programs (or an application based on these programs).

 

 

Quick Steps

 

1) make_sil_sa_dir or make_sil_sa_bundle

2) make STATIC=1

3) register static library in the SIL-SA application

 

 

There is no limit to the number of static SIL-SA libraries that can be linked into the program image or registered with the SIL-SA subsystem.

 

A static SIL-SA library is not used unless it is configured for use in the main server. The code is available at all times (since it is statically linked) and cannot be removed (only disabled).

 

Enable a static SIL-SA library

Disable a static SIL-SA library

10.5.1  Static SIL-SA Library Example

 

This information is also in the README-STATIC_SILSA.txt file in the root of the source code directory.

 

 

 

IMPORTANT info on Using Static SIL-SA Libraries

==============================================================================

 

0) Assume a SIL-SA library is setup

 

 > make_sil_sa_dir test2

 

This could also be a bundle (make_sil_sa_bundle)

 

1) Build a static version of the SIL-SA library

 

 > cd test2

 > make STATIC=1

 

2) Copy the static library to a common directory if desired.

   This is not required but may make using the STATIC_SILSA variable easier.

 

 > cp lib/libtest2.a $HOME/silsa/libtest2_sa.a

 

3) Use the macro STATIC_SILSA when building the sil-sa-app

 

Option 1: Change in Makefile:

 

  STATIC_SILSA=-L /home/andy/silsa -ltest2_sa

 

Option 2: set from command line:

 

  > STATIC_SILSA='-L /home/andy/silsa -ltest2_sa' make

 

Note that the -L part must be first and it must specify where the static

libraries are located.  One or more -l parts can follow.

The -l parameter does not use the full name. Instead libtest2_sa.a can

be specified as -ltest2_sa.

 

4) Register the static library in the sil-sa-app or combo-app program

 

Example from sil-sa-app/main.c:

 

/* extern definitions for the 3 expected callbacks */

AGT_SIL_LIB_EXTERN(test2)

 

static status_t static_silsa_init (void)

{

    /* example: module=test2;

     * need to use in Makefile (example)

     * STATIC_SILSA=-L /home/andy/silsa -ltest2_sa

     * The actual library names are not needed in this code

     */

    status_t res =

        agt_sil_lib_register_statlib((const xmlChar *)"test2",

                                     y_test2_init,

                                     y_test2_init2,

                                     y_test2_cleanup);

 

    return res;

}

 

5) The SIL-SA library will not be used unless the module or bundle is loaded.

 

   > load test2

   > netconfd-pro module=test2

 

10.6  SIL Makefile

The SIL-SA Makefile is based on the Makefile for YumaPro sources.  The automake program is not used at this time.  There is no ./configure script to run.  There are 2 basic build steps:

  1. make [DEBUG=1] [STATIC=1]

  2. [sudo] make install

The installation step may require root access via the sudo program, depending on the system.

 

10.6.1  Target Platforms

The following target platforms are supported:

10.6.2  Build Targets

The following table describes the build targets that are supported:

 

YumaPro Build Targets

 

Make Target

Description

all

Make everything.  This is the default if not target is specified.

depend

Make the dependencies files.

clean

Remove the target files.

superclean

Remove all the dependencies and the target files.

distclean

Remove all the distribution files that make be installed, all the dependencies and the target files.

test

Make any test code.

lint

Run a lint program on the source files.

install

Install the components into the system.

uninstall

Remove the components from the system.

 

10.6.3  Command Line Build Options

There are several command line options that can be used when making the YumaPro source code.  These may have an impact on building and linking the SIL source code.  The top-level README file in the source code contains the latest definition of the make flags.

 

10.6.4  Example SIL Makefile

The script /usr/bin/make_sil_dir_pro or /usr/bin/make_sil_bundle is used to automatically create a SIL work sub-directory tree.  Support files are located in the /usr/share/yumapro/util directory.

The Makefiles for SIL and SIL-SA modules are located in the “util” directory

 

 

10.7  Controlling SIL Behavior

There are several extensions and API functions that control how the server will invoke the SIL code and process data in edit requests

10.7.1  SIL Invocation Order (sil-priority)

It is possible for the client to send several edits within a single <edit-config> or <copy-config> request.

Sometimes it is useful to make sure certain data structures are validated and processed before other data structures.

The sil-priority extension can be used to control the order that edits are processed. This extension is defined in section 12.2 “Built-in YANG Language Extensions”.

 

The SIL priority can also be set at run-time with an API function.  This allows SIL priority to be setup in platform-specific ways, without needing to alter the YANG file for this purpose.

 

 

/********************************************************************

* FUNCTION agt_cb_set_sil_priority

*

* Set the desired SIL priority with a callback instead of

* using the YANG extension for this purpose

*

* INPUTS:

*   defpath == Xpath with default (or no) prefixes

*   sil_priority = SIL priority value to use (1..255)

* RETURNS:

*   status

*********************************************************************/

extern status_t

    agt_cb_set_sil_priority (const xmlChar *defpath,

                             uint8 sil_priority);

 

 

Example: Set the SIL priority to 30 for the /interfaces/interface list.

 

 

    status_t res =
     agt_cb_set_sil_priority((const xmlChar *)”/if:interfaces/if:interface”,

                              30);

 

 

10.7.2  Reverse SIL priority for delete operations (--sil-prio-reverse-for-deletes)

It is possible for the client to send several edits within a single <edit-config> request. Sometimes it is useful to make sure certain data structures are validated and processed before other data structures. The "sil-priority" extension can be used to control the order of the edits and how they are processed.

In a case where you set "sil-priority" for multiple specific objects and want the order to be reversed during the DELETE operations there is a netconfd-pro configuration parameter "sil-prio-reverse-for-deletes".

If set to "true" the "sil-prio-reverse-for-deletes" parameter for netconfd-pro dictates that the SIL priority for DELETE operations will be reversed. This parameter can be used to delete leafref nodes with referenced by node in reverse order.
If 'false' then the SIL priority will not be reversed. The default is "false".

When the parameter is set to "true" the server will invoke callbacks for the specific objects with "sil-prority" configured in reverse order.

The "sil-prio-reverse-for-deletes" parameter has following facts that should be noted:

Please note that the EDIT2 MERGE edits will not change on the invocation order if the callbacks have mixed operations on children (if there are delete on one child and other edit operation on second child), they will not be reversed anyhow.

The EDIT2 MERGE mode combines multiple edits in one callback and the server invoke just one callback for all the edits. Hence, the server has no any control of when it should reverse the priority for this callback in case of mixed children operations. The priority in this case will remain the same and the server will warn a user with following warning:

 

 

SIL priority for object 'uint16-leaf' set to 100.100

Warning: Cannot reverse SIL priority for EDIT2 child 'test:uint16-leaf'

 

 

However, in case the EDIT2 MERGE mode has only delete or remove operations on children the priority for this callback will be reversed.

It is recommended to use EDIT1 callbacks to fully control the reverse priority behavior. Or as an alternative it is recommended to avoid the EDIT2 MERGE mode with mixed operations if you want to reverse the priority of the edits.

Also, the server will reverse the edits priorities in case the EDIT2 children edit is the edit to delete or remove the default node. And if there is no any other children edits with not equal to remove or delete, or merge edit operation in case of default node removal.

 

10.7.3  Deletion of Child Nodes (sil-delete-children-first)

If the client deletes a container of list entries, then the server will normally only invoke the SIL callback for deletion for the container.  It may be easier to allow the server to use the SIL callbacks for the child list nodes, to delete a data structure “bottom-up”.

Use the sil-delete-children-first extension to force the server to delete all the child nodes before deleting the parent container.

The sil-delete-children-first extension can be used in any parent node (container or list) and can be used to force the server to invoke the SIL callback for any child node (container, list, leaf, leaf-list, anyxml).

 

10.7.4  Suppress leafref Validation

Leafref validation involves searching all the “pointed at” leaf instances to make sure that the leafref leaf being validated contains one of the values actually in use.  An instance is required to exist for the leafref to be valid.

YANG 1.0 does not support the “require-instance” sub-statement for a leafref definition.  In order to force the server to skip this validation step, use the agt_cb_skip_leafref_validation API function.

 

 

/********************************************************************

* FUNCTION agt_cb_skip_leafref_validation

*

* Set a previously registered callback as a node

* that the server should skip leafref validation

* in order to save processing time

*

* INPUTS:

*   defpath == Xpath with default (or no) prefixes

*

* RETURNS:

*   status

*********************************************************************/

extern status_t

    agt_cb_skip_leafref_validation (const xmlChar *defpath);

 

 

 

Specify the path for the leafref node that should have validation skipped.

In the following example, the path “/t641-1:B/t641-1:BB/t641-1:fd-mode” represents a leafref leaf that will be treated as if the “require-instance false” statement was present in the YANG definition:

 

 

Inside SIL init function:

 

 

    res = agt_cb_register_callback(

        y_test_fd641_1_M_test_fd641_1,

        (const xmlChar *)"/t641-1:B/t641-1:BB/t641-1:fd-mode",

        y_test_fd641_1_R_test_fd641_1,

        test_fd641_1_B_BB_fd_mode_edit);

    if (res != NO_ERR) {

        return res;

    }

 

    res = agt_cb_skip_leafref_validation(

        (const xmlChar *)"/t641-1:B/t641-1:BB/t641-1:fd-mode");

    if (res != NO_ERR) {

        return res;

    }

 

 

 

 

 

10.8 Short Names for SIL and SIL-SA Code Generation

Starting in release 20.10-5 the short-names” parameter is available to control long or short name generation for SIL and SIL-SA code.  

 

Example:

 

 

Path: leaf /interfaces/interface/admin-status

 

GET2 long function: u_ietf_interfaces_interfaces_interface_admin_status_get

 

GET2 short function: u_if_admin_status_get

 

 

10.8.1  CLI Parameters

There are two yangdump-sdk (or make_sil* script) CLI parameters that control the short name generation

 

 

 

        leaf force-prefix {

          description

            "If the 'short-names' parameter is 'true' then this

             object can be used to force a specific prefix

             value instead of the module prefix, when generating

             instrumentation code.

 

             This object should only be used if the module prefix is

             known to cause a naming conflict with existing code.

 

             This object cannot be used if the 'sil-bundle' parameter

             is also used, and --short-names=true.";

          type yt:NcxName;

        }

 

 

        leaf short-names {

          description

            "If 'true', generate instrumentation code using short

             names whenever possible. The module prefix and the

             node name will be used. A numeric qualifier will be

             appended to the name if the short name would cause

             a duplicate symbol to be generated. The 'force-prefix'

             value will be use for the prefix if that parameter is

             present.

 

             If 'false', then generate instrumentation code using

             long names which encode the module name and the

             entire path to the object into the name.";

          type boolean;

          default true;

        }

 

 

 

10.8.2 Short Name Summary

If the --short-names parameter is set to “true” (the default value) then short names will be used.

The header for each C file will contain the Short Name Mapping used.

 

SIL Example:

 

 

> make_sil_dir_pro --split ietf-interfaces --sil-get2 --sil-edit2

 

 

 

Example Short Name Summary from u_ietf-interfaces.c

 

 

  Short Name Mappings

    admin_status = /interfaces/interface/admin-status

    admin_status_1 = /interfaces-state/interface/admin-status

    description = /interfaces/interface/description

    discontinuity_time = /interfaces/interface/statistics/discontinuity-time

    discontinuity_time_1 = /interfaces-state/interface/statistics/discontinuity-time

    enabled = /interfaces/interface/enabled

    higher_layer_if = /interfaces/interface/higher-layer-if

    higher_layer_if_1 = /interfaces-state/interface/higher-layer-if

    if_index = /interfaces/interface/if-index

    if_index_1 = /interfaces-state/interface/if-index

    in_broadcast_pkts = /interfaces/interface/statistics/in-broadcast-pkts

    in_broadcast_pkts_1 = /interfaces-state/interface/statistics/in-broadcast-pkts

    in_discards = /interfaces/interface/statistics/in-discards

    in_discards_1 = /interfaces-state/interface/statistics/in-discards

    in_errors = /interfaces/interface/statistics/in-errors

    in_errors_1 = /interfaces-state/interface/statistics/in-errors

    in_multicast_pkts = /interfaces/interface/statistics/in-multicast-pkts

    in_multicast_pkts_1 = /interfaces-state/interface/statistics/in-multicast-pkts

    in_octets = /interfaces/interface/statistics/in-octets

    in_octets_1 = /interfaces-state/interface/statistics/in-octets

    in_unicast_pkts = /interfaces/interface/statistics/in-unicast-pkts

    in_unicast_pkts_1 = /interfaces-state/interface/statistics/in-unicast-pkts

    in_unknown_protos = /interfaces/interface/statistics/in-unknown-protos

    in_unknown_protos_1 = /interfaces-state/interface/statistics/in-unknown-protos

    interface = /interfaces/interface

    interface_1 = /interfaces-state/interface

    interfaces = /interfaces

    interfaces_state = /interfaces-state

    last_change = /interfaces/interface/last-change

    last_change_1 = /interfaces-state/interface/last-change

    link_up_down_trap_enable = /interfaces/interface/link-up-down-trap-enable

    lower_layer_if = /interfaces/interface/lower-layer-if

    lower_layer_if_1 = /interfaces-state/interface/lower-layer-if

    name = /interfaces/interface/name

    name_1 = /interfaces-state/interface/name

    oper_status = /interfaces/interface/oper-status

    oper_status_1 = /interfaces-state/interface/oper-status

    out_broadcast_pkts = /interfaces/interface/statistics/out-broadcast-pkts

    out_broadcast_pkts_1 = /interfaces-state/interface/statistics/out-broadcast-pkts

    out_discards = /interfaces/interface/statistics/out-discards

    out_discards_1 = /interfaces-state/interface/statistics/out-discards

    out_errors = /interfaces/interface/statistics/out-errors

    out_errors_1 = /interfaces-state/interface/statistics/out-errors

    out_multicast_pkts = /interfaces/interface/statistics/out-multicast-pkts

    out_multicast_pkts_1 = /interfaces-state/interface/statistics/out-multicast-pkts

    out_octets = /interfaces/interface/statistics/out-octets

    out_octets_1 = /interfaces-state/interface/statistics/out-octets

    out_unicast_pkts = /interfaces/interface/statistics/out-unicast-pkts

    out_unicast_pkts_1 = /interfaces-state/interface/statistics/out-unicast-pkts

    phys_address = /interfaces/interface/phys-address

    phys_address_1 = /interfaces-state/interface/phys-address

    speed = /interfaces/interface/speed

    speed_1 = /interfaces-state/interface/speed

    statistics = /interfaces/interface/statistics

    statistics_1 = /interfaces-state/interface/statistics

    type = /interfaces/interface/type

    type_1 = /interfaces-state/interface/type

 

 

 

10.8.3 Constants

The constants generated in a SIL or SIL-SA H file will be affected by the short-names parameter.

 

Examples

 

Module and Revision Constants:

Long name constant:

 

#define y_ietf_interfaces_M_ietf_interfaces (const xmlChar *)"ietf-interfaces"

 #define y_ietf_interfaces_R_ietf_interfaces (const xmlChar *)"2018-02-20"

 

Short name constant:

 

#define y_if_M_if (const xmlChar *)"ietf-interfaces"

#define y_if_R_if (const xmlChar *)"2018-02-20"

 

 

Feature Constants

Long name constant:

 

#define u_ietf_interfaces_F_arbitrary_names 1

 

Short name constant:

 

#define u_if_F_arbitrary_names 1

 

 

Data Structure Typedef Constants:

Long name constant:

 

/* leaf-list /interfaces/interface/lower-layer-if */

typedef struct y_ietf_interfaces_T_interfaces_interface_lower_layer_if_ {

    dlq_hdr_t qhdr;

    xmlChar *v_lower_layer_if;

} y_ietf_interfaces_T_interfaces_interface_lower_layer_if;

 

Short name constant:

 

typedef struct y_if_T_lower_layer_if_ {

    dlq_hdr_t qhdr;

    xmlChar *v_lower_layer_if;

} y_if_T_lower_layer_if;

 

 

Object Name Constant:

Long name constant:

 

#define y_ietf_interfaces_N_admin_status (const xmlChar *)"admin-status"

 

Short name constant:

 

#define y_if_N_admin_status (const xmlChar *)"admin-status"

 

 

 

10.8.4 Function and Variable Names

The function and variable names generated in a SIL or SIL-SA H file will be affected by the short-names parameter.

 

GET2 Example:

Long name:

 

extern status_t u_ietf_interfaces_interfaces_interface_admin_status_get (

    getcb_get2_t *get2cb,

    const xmlChar *k_interfaces_interface_name);

 

Short name:

 

extern status_t u_if_admin_status_get (

    getcb_get2_t *get2cb,

    const xmlChar *k_name);

 

 

 

 

EDIT2 Example:

 

 

 

Long name:

 

extern status_t u_ietf_interfaces_interfaces_interface_edit (

    ses_cb_t *scb,

    rpc_msg_t *msg,

    agt_cbtyp_t cbtyp,

    op_editop_t editop,

    val_value_t *newval,

    val_value_t *curval,

    const xmlChar *k_interfaces_interface_name);

 

Short name:

 

extern status_t u_if_interface_edit (

    ses_cb_t *scb,

    rpc_msg_t *msg,

    agt_cbtyp_t cbtyp,

    op_editop_t editop,

    val_value_t *newval,

    val_value_t *curval,

    const xmlChar *k_name);