neuray API Programmer's Manual

mi::neuraylib::INode_manager_client Class Reference

[Node manager]

Description

The node manager client allows to start or join clusters built from worker nodes. It should be used in a process running on a client node.

Public Member Functions

virtual void add_shutdown_cluster_callback( IShutdown_cluster_callback* callback) =0
Adds a callback to be called when a request to shutdown a cluster is received. More...
virtual void add_shutdown_node_managers_callback( IShutdown_node_managers_callback* callback) =0
Adds a callback to be called when a request to shutdown all clients and workers is received. More...
virtual void add_worker_process_started_callback( IWorker_process_started_callback* callback) =0
Adds a callback to be called when a worker process has been fully started. More...
virtual const ICluster_descriptorget_cluster( Size index) const =0
Returns a descriptor for a cluster. More...
virtual const IStringget_head_node_interface() =0
Get the head node address and subnet qualifier. More...
virtual const IStringget_listen_address() const =0
Returns the listen address used by the node manager. More...
virtual const IStringget_multicast_base_address() const =0
Returns the multicast base address. More...
virtual Size get_number_of_clusters() const =0
Returns the number of existing clusters.
virtual Size get_number_of_worker_nodes() const =0
Returns the number of worker nodes currently known to the node manager.
virtual const IWorker_node_descriptorget_worker_node( Size index) const =0
Returns a descriptor for a worker node currently known to the node manager. More...
virtual INode_manager_clusterjoin_or_create_cluster( Size min_number_of_requested_worker_nodes, Size max_number_of_requested_worker_nodes, ICluster_filter* cluster_filter, IWorker_node_filter* worker_node_filter, const char* program_name, const char* argument_string, bool reusable = false, Uint32 child_process_timeout = 0) =0
Joins an existing cluster or creates a new one. More...
virtual void remove_shutdown_cluster_callback( IShutdown_cluster_callback* callback) =0
Removes a previously added callback to be called when a request to shutdown a cluster is received. More...
virtual void remove_shutdown_node_managers_callback( IShutdown_node_managers_callback* callback) =0
Removes a previously added callback to be called when a request to shutdown all clients and workers is received. More...
virtual void remove_worker_process_started_callback( IWorker_process_started_callback* callback) =0
Removed a callback to be called when a worker process has been fully started. More...
virtual void set_head_node_interface( const char* address) =0
Set the head node address and subnet qualifier. More...
virtual Sint32 set_multicast_base_address( const char* base_address) =0
Sets the multicast base address. More...
virtual Sint32 shutdown() =0
Shuts down the operation of the node manager. More...
virtual Sint32 shutdown_cluster( const ICluster_descriptor* cluster_descriptor) =0
Unconditionally shut down the cluster that cluster_descriptor refers to and release reserved worker nodes. More...
virtual void shutdown_node_managers() =0
Shuts down all node manager clients and workers which are currently joined. More...
virtual Sint32 start( const char* listen_address, bool tcp = false, const char* discovery_address = 0, const char* cluster_interface = 0) =0
Starts the operation of the node manager. More...

Static Public Variables

static const Uint32 SIGNAL_STARTUP_ONLY = 0xffffffff
Symbolic constant to pass as child_process_timeout to join_or_create_cluster. More...

Member Functions

virtual void mi::​neuraylib::​INode_manager_client::add_shutdown_cluster_callback( IShutdown_cluster_callback* callback) [pure virtual]

Adds a callback to be called when a request to shutdown a cluster is received.

See also:

remove_shutdown_cluster_callback()

Parameters

callback
The callback to be added.
virtual void mi::​neuraylib::​INode_manager_client::add_shutdown_node_managers_callback( IShutdown_node_managers_callback* callback) [pure virtual]

Adds a callback to be called when a request to shutdown all clients and workers is received.

See also:

remove_shutdown_node_managers_callback()

Parameters

callback
The callback to be added.
virtual void mi::​neuraylib::​INode_manager_client::add_worker_process_started_callback( IWorker_process_started_callback* callback) [pure virtual]

Adds a callback to be called when a worker process has been fully started.

See also:

remove_worker_process_started_callback()

Parameters

callback
The callback to be added.
virtual const ICluster_descriptor* mi::​neuraylib::​INode_manager_client::get_cluster( Size index) const [pure virtual]

Returns a descriptor for a cluster.

Note:

The set of clusters can change at any time. This function can therefore return NULL even if index is smaller than the result of the last call to get_number_of_clusters().

Parameters

index
The index of the cluster (from 0 to get_number_of_clusters()-1).

Returns

The descriptor for the specified cluster, or NULL if index is out of bounds.

virtual const IString* mi::​neuraylib::​INode_manager_client::get_head_node_interface() [pure virtual]

Get the head node address and subnet qualifier. This returns the address set using set_head_node_interface().

Returns

The head node interface address and subnet qualifier.

virtual const IString* mi::​neuraylib::​INode_manager_client::get_listen_address() const [pure virtual]

Returns the listen address used by the node manager.

Returns

The listen address and port.

virtual const IString* mi::​neuraylib::​INode_manager_client::get_multicast_base_address() const [pure virtual]

Returns the multicast base address. The node manager reserves a unique multicast address for each cluster that it manages. These multicast addresses start at the multicast base address and are obtained by incrementing the last octet, then the second-last octet, and so on.

See also:

set_multicast_base_address(), mi::neuraylib::ICluster_descriptor::get_multicast_address()

Returns

The currently set multicast base address.

virtual Size mi::​neuraylib::​INode_manager_client::get_number_of_clusters() const [pure virtual]

Returns the number of existing clusters.

virtual Size mi::​neuraylib::​INode_manager_client::get_number_of_worker_nodes() const [pure virtual]

Returns the number of worker nodes currently known to the node manager.

virtual const IWorker_node_descriptor* mi::​neuraylib::​INode_manager_client::get_worker_node( Size index) const [pure virtual]

Returns a descriptor for a worker node currently known to the node manager.

Note:

The set of worker nodes in the cluster can change at any time. That is, this function can return NULL even if index is smaller than the result of the last call to get_number_of_worker_nodes().

Parameters

index
The index of the worker node (from 0 to get_number_of_worker_nodes()-1).

Returns

The descriptor for the specified worker node, or NULL if index is out of bounds.

virtual INode_manager_cluster* mi::​neuraylib::​INode_manager_client::join_or_create_cluster( Size min_number_of_requested_worker_nodes, Size max_number_of_requested_worker_nodes, ICluster_filter* cluster_filter, IWorker_node_filter* worker_node_filter, const char* program_name, const char* argument_string, bool reusable = false, Uint32 child_process_timeout = 0) [pure virtual]

Joins an existing cluster or creates a new one. This function will do the following:

  • For all existing clusters:
    • Check if the cluster size matches the minimum and maximum number of requested worker nodes.

    • Check if the program name and argument string of the cluster nodes match the values given in program_name and argument_string.

    • Check if calling the cluster_filter callback returns true.

    • If all above conditions are met, the calling node joins the existing cluster as a new client node. An interface describing the existing cluster is returned. The function terminates.

  • If no matching cluster was found, then for all existing worker nodes:
    • Check if worker_node_filter returns true, if yes reserve the worker node. Leave the loop if the number of reserved worker nodes meets the maximum number of requested nodes.

  • If at least the minimum number of requested worker nodes have been reserved, create the cluster, and an interface describing the newly created cluster is returned.

  • Otherwise, the reservation of worker nodes is released, and NULL is returned.

If the flag child_process_watchdog is set to true, the child process started on worker nodes will be under closer scrutiny by the node manager. The node manager will substitute any occurrence of the placeholder w in the command line used to start the child process with the string identifier of the named pipe the child process is supposed to write to. The child process may open the pipe for writing and may write PDUs from the watchdog protocol to it. The watchdog is activated, once the first PDU has been written to the pipe. The child process needs to keep writing PDUs in intervals of n seconds from that moment on or the node manager will give up on the child process and will terminate it.

Parameters

min_number_of_requested_worker_nodes
The minimum number of worker nodes expected in the cluster.
max_number_of_requested_worker_nodes
The maximum number of worker nodes expected in the cluster.
cluster_filter
A filter specifying required cluster properties. If NULL, no existing cluster will be joined.
worker_node_filter
A filter specifying required worker node properties. If NULL, no cluster will be created. The filter will be used again when calling mi::neuraylib::INode_manager_cluster::grow() to add nodes to the cluster later on.
program_name
The name of the program to run on the worker nodes. If NULL, a cluster is created without child processes being forked by the worker nodes.
argument_string
Arguments to program_name. The string may include the substring %m which will be replaced by the multicast address of the cluster (use %% to escape percent signs). It may also contain the substring %h which will be expanded into the head node, a node marked within the cluster for applications that need support for it. The %H substring will be expanded to 1 on the head node, to 0 on all other hosts. The %n substring will be expanded to the number of nodes in the cluster. If the child_process_timeout parameter is defined, the command line also needs to contain the substrings %w, which will be substituted with the name of the named pipe and %t which will be replaced with the timeout value. If a substring of the format %p:property is included, it will be expanded to the value for the property set by the worker. Please note that adding %p placeholders dismisses nodes during recruitment that do not have the assigned property set.
reusable
A flag indicating whether it should be allowed to start a different child process application after having shut down the application that was run in the cluster first. Clusters flagged 'reusable' can't be joined by more than one client. This flag is only useful if new child processes are supposed to be started using mi::neuraylib::INode_manager_cluster::start_worker_program().
child_process_timeout
A parameter indicating whether there should be a watchdog controlling the child process on worker nodes. The value specifies the timeout in seconds after which the node manager assumes the child process is hanging. It needs to be >= 5 seconds. This parameter only has an effect when creating new clusters. If it is defined, the argument_string parameter needs to contain the substrings %w and %t.

Returns

An interface to the joined or created cluster, or NULL in case of failure.

virtual void mi::​neuraylib::​INode_manager_client::remove_shutdown_cluster_callback( IShutdown_cluster_callback* callback) [pure virtual]

Removes a previously added callback to be called when a request to shutdown a cluster is received.

See also:

add_shutdown_cluster_callback()

Parameters

callback
The callback to be removed.
virtual void mi::​neuraylib::​INode_manager_client::remove_shutdown_node_managers_callback( IShutdown_node_managers_callback* callback) [pure virtual]

Removes a previously added callback to be called when a request to shutdown all clients and workers is received.

See also:

add_shutdown_node_managers_callback()

Parameters

callback
The callback to be removed.
virtual void mi::​neuraylib::​INode_manager_client::remove_worker_process_started_callback( IWorker_process_started_callback* callback) [pure virtual]

Removed a callback to be called when a worker process has been fully started.

See also:

add_worker_process_started_callback()

Parameters

callback
The callback to be removed.
virtual void mi::​neuraylib::​INode_manager_client::set_head_node_interface( const char* address) [pure virtual]

Set the head node address and subnet qualifier. The IP address of the interface can be specified as a sub net using the CIDR notation a.b.c.d/xx. If there is an interface on the host with an address inside this range the first match will be used in the address returned by mi::neuraylib::IWorker_node_descriptor::get_address() for the worker node descriptor returned by mi::neuraylib::INode_manager_cluster::get_head_node. This means that on a host which has the address 192.168.1.1, specifying the address as 192.168.0.0/16 would return the address 192.168.1.1.

To have an effect, this needs to be set before creating the cluster.

virtual Sint32 mi::​neuraylib::​INode_manager_client::set_multicast_base_address( const char* base_address) [pure virtual]

Sets the multicast base address. The node manager reserves a unique multicast address for each cluster that it manages. These multicast addresses start at the multicast base address and are obtained by incrementing the last octet, then the second-last octet, and so on.

See also:

get_multicast_base_address(), mi::neuraylib::ICluster_descriptor::get_multicast_address()

Parameters

base_address
A multicast address to be used as a base when reserving cluster multicast addresses.

Returns

0 in case of success, -1 otherwise

virtual Sint32 mi::​neuraylib::​INode_manager_client::shutdown() [pure virtual]

Shuts down the operation of the node manager.

Returns

0 in case of success, -1 otherwise.

virtual Sint32 mi::​neuraylib::​INode_manager_client::shutdown_cluster( const ICluster_descriptor* cluster_descriptor) [pure virtual]

Unconditionally shut down the cluster that cluster_descriptor refers to and release reserved worker nodes. The caller does not need to be the creator of the cluster.

Parameters

cluster_descriptor
Descriptor of the cluster that is supposed to be shut down.

Returns

0 in case of success, -1 if cluster_descriptor is unknown.

virtual void mi::​neuraylib::​INode_manager_client::shutdown_node_managers() [pure virtual]

Shuts down all node manager clients and workers which are currently joined. Can be used for example when installing new versions.

virtual Sint32 mi::​neuraylib::​INode_manager_client::start( const char* listen_address, bool tcp = false, const char* discovery_address = 0, const char* cluster_interface = 0) [pure virtual]

Starts the operation of the node manager.

Parameters

listen_address
The address used to communicate with other node manager instances. Should be a multicast address unless TCP networking is selected. In case of TCP networking, if the address is the local IP address, the host will become the head node which is used to discover the other hosts.
tcp
Indicates whether TCP or UDP should be used.
discovery_address
The address of the TCP head node used for host discovery. If this is the same as the listen address, the node will be head node.
cluster_interface
The address of the cluster interface for listening.

Returns

0 in case of success, -1 otherwise

Variables

const Uint32 mi::​neuraylib::​INode_manager_client::SIGNAL_STARTUP_ONLY = 0xffffffff [static]

Symbolic constant to pass as child_process_timeout to join_or_create_cluster. If set, the watchdog mechanism is disabled and only signals when the child process has been fully started.

See also:

join_or_create_cluster()