REQUIRE System - Dynamic Library Loading

The REQUIRE statement enables hot-loading of JavaScript libraries at runtime, supporting all execution modes (Autonomous Web, Controlled Web, Command-line) with automatic dependency resolution.

Basic Syntax

REQUIRE "github.com/<owner>/<library>@<tag>"
REQUIRE "central:<owner>/<library>@<tag>"
REQUIRE "./relative/path/to/library.js"
REQUIRE "../shared/utils.js" 
REQUIRE "github.com/<owner>/<library>@SNAPSHOT"

Examples:

-- Load specific released version (direct GitHub)
REQUIRE "github.com/alice/math-utils@v1.2.3"

-- Load registry-verified version with integrity checking
REQUIRE "central:alice/math-utils@v1.2.3"

-- Load latest released version  
REQUIRE "github.com/bob/chart-lib@latest"

-- Load development version (always fetches latest)
REQUIRE "github.com/alice/math-utils@SNAPSHOT"

-- Load local relative file (development)
REQUIRE "./libs/my-local-utils.js"
REQUIRE "../shared-components/chart-helpers.js"

-- Use loaded functions (same regardless of source)
LET result = CALCULATE_STATISTICS data=[1,2,3,4,5]
LET chart = CREATE_LINE_CHART data=result.values

Environment Detection and Loading

Browser Environment

Network requests via fetch():

Libraries loaded via HTTP requests to GitHub
Code executed using eval() in global scope
Functions registered in window namespace
Library detection via window['library-name']

Example flow:

// 1. Fetch library code
fetch('https://github.com/alice/math-utils/releases/download/v1.2.3/math-utils-min.js')
  
// 2. Execute in global scope
eval(libraryCode)

// 3. Functions now available globally
window['math-utils'].CALCULATE_STATISTICS([1,2,3])

Node.js Environment

Network requests via https.get():

Same GitHub URLs, different HTTP client
Code executed using vm.runInContext() with proper global context
Functions registered in created context + copied to global scope
Library detection via global['library-name']

Example flow:

// 1. Fetch library code  
https.get('https://github.com/alice/math-utils/releases/download/v1.2.3/math-utils-min.js', ...)

// 2. Execute in isolated context with global access
const context = vm.createContext({ ...global });
vm.runInContext(libraryCode, context);

// 3. Copy functions to main global scope
global['math-utils'] = context['math-utils'];

Library Loading Sequence

Fallback GET Sequence

For REQUIRE "github.com/<owner>/<libName>@<tag>":

1st attempt - GitHub Releases (Minified) ⭐:

GET https://github.com/<owner>/<libName>/releases/download/<tag>/<libName>-min.js

2nd attempt - GitHub Releases (Development) ⭐:

GET https://github.com/<owner>/<libName>/releases/download/<tag>/<libName>.js

3rd attempt - GitHub Releases (Versioned fallback):

GET https://github.com/<owner>/<libName>/releases/download/<tag>/<libName>-<tag>.js

4th attempt - GitHub Releases (Generic bundle):

GET https://github.com/<owner>/<libName>/releases/download/<tag>/bundle.js

5th attempt - GitHub Releases (Generic index):

GET https://github.com/<owner>/<libName>/releases/download/<tag>/index.js

6th attempt - GitHub Raw Files (Development fallback):

GET https://raw.githubusercontent.com/<owner>/<libName>/<tag>/libs/<libName>.js

First successful GET wins and stops the sequence.

Development Workflows

Local File References

For libraries under development that aren’t released yet:

-- Relative paths (resolved from current script location)
REQUIRE "./my-development-lib.js"           -- Same directory
REQUIRE "../shared/common-utils.js"         -- Parent directory  
REQUIRE "../../team-libs/specialized.js"    -- Multiple levels up
REQUIRE "./subfolder/helpers.js"            -- Subdirectory

Node.js Development Mode (Command Line)

Full compatibility with Node.js ecosystem:

-- Node.js modules work directly via native require()
REQUIRE "./utils.js"              -- Uses Node.js require() automatically
REQUIRE "../shared/helpers.js"    -- Node.js path resolution
REQUIRE "lodash"                  -- npm packages work seamlessly

Loading behavior in Node.js mode:

Native Node.js require() - ./relative/paths.js use Node.js module loading
Full module.exports support - Existing Node.js modules work instantly
Node.js path resolution - Relative to current script file
No build step required - Direct execution of development code
Mixed dependencies - Node.js modules + GitHub libraries seamlessly
Hot reloading - File changes reload automatically
Full npm ecosystem - All Node.js packages available via require()

Example Node.js module (works as-is):

// shared/math-helpers.js - Standard Node.js module
function calculateAverage(numbers) {
  return numbers.reduce((a, b) => a + b) / numbers.length;
}

function findMax(numbers) {
  return Math.max(...numbers);
}

module.exports = { calculateAverage, findMax };

Usage in RexxJS (Node.js mode):

-- Loads Node.js module automatically
REQUIRE "./shared/math-helpers.js"

-- Node.js functions auto-wrapped as RexxJS functions
LET avg = CALCULATEAVERAGE data=[1,2,3,4,5]
LET max = FINDMAX data=[1,2,3,4,5]
SAY "Average:" avg "Max:" max

Browser Development Mode (Webpack Required)

Webpack preprocessing required for local files:

-- Browser mode - local files must be webpack-bundled
REQUIRE "https://localhost:8080/dist/my-dev-bundle.js"    -- Bundled local code
REQUIRE "https://localhost:8080/dist/shared-bundle.js"    -- Bundled shared code
REQUIRE "github.com/alice/math-utils@latest"              -- GitHub libraries work directly

Loading behavior in browser mode:

No native require() - Browser cannot resolve ./relative/paths.js
Webpack bundling required - Local files must be pre-processed
HTTP/HTTPS URLs only - All resources must be web-accessible
Build step necessary - Development workflow requires build process
GitHub libraries work directly - No build step needed for external libraries

Browser development workflow:

# 1. Bundle local development files
webpack-dev-server --mode=development

# 2. RexxJS script can now load bundled code
# REQUIRE "http://localhost:8080/dist/my-feature-bundle.js"

Development Environment Comparison

Feature	Node.js Mode	Browser Mode
Local files	✅ Direct `./file.js`	❌ Must bundle first
Node.js modules	✅ Native require()	❌ Must bundle first
npm packages	✅ Direct access	❌ Must bundle first
GitHub libraries	✅ Direct REQUIRE	✅ Direct REQUIRE
Hot reloading	✅ Automatic	⚠️ Via webpack-dev-server
Build step	❌ Not needed	✅ Required
Debugging	✅ Native Node.js	⚠️ Browser DevTools

Recommendation for development:

Start in Node.js mode for rapid development and testing
Switch to browser mode for final testing and deployment

Local Library Development Workflow

Phase 1: Local Development (Node.js)

-- Direct development with Node.js modules
REQUIRE "./my-new-library.js"        -- Your library under development
REQUIRE "../shared/team-utils.js"    -- Shared team modules
REQUIRE "lodash"                     -- npm packages
REQUIRE "github.com/alice/math-utils@latest"  -- External GitHub libraries

-- Test your library
LET result = MY_NEW_FUNCTION data=[1,2,3]
SAY "Development result:" result

Phase 2: Pre-Release Testing (SNAPSHOT)

# Publish development version to GitHub
gh release create SNAPSHOT my-library.js

# Update RexxJS scripts
# REQUIRE "github.com/me/my-library@SNAPSHOT"

Phase 3: Browser Compatibility (Webpack)

# Bundle for browser testing
webpack --mode=development --output-filename=my-library-bundle.js

# Test in browser
# REQUIRE "http://localhost:8080/dist/my-library-bundle.js"

Phase 4: Production Release

# Create release
gh release create v1.0.0 my-library.js my-library-min.js

# Update production scripts
# REQUIRE "github.com/me/my-library@v1.0.0"

Example local development library:

/*!
 * my-dev-lib v0.0.0-dev | Development Version
 * @rexxjs-meta {
 *   "dependencies": {
 *     "github.com/alice/math-utils": "latest",
 *     "./other-local-lib.js": "dev"
 *   }
 * }
 */

const myDevLib = {
  'MY_DEV_LIB_MAIN': () => ({
    type: 'library_info',
    name: 'My Development Library',
    version: '0.0.0-dev',
    loaded: true
  }),
  
  'EXPERIMENTAL_FUNCTION': (data) => {
    // Cutting-edge features under development
  }
};

// Standard dual export
if (typeof module !== 'undefined' && module.exports) {
  module.exports = { 'my-dev-lib': myDevLib };
  if (typeof global !== 'undefined') global['my-dev-lib'] = myDevLib;
} else if (typeof window !== 'undefined') {
  window['my-dev-lib'] = myDevLib;
}

SNAPSHOT Versions

For libraries with ongoing development builds (Maven-style):

-- Always fetches latest development version
REQUIRE "github.com/alice/math-utils@SNAPSHOT"

SNAPSHOT loading behavior:

Skips caching - Always fetches fresh version

Tries SNAPSHOT-specific URLs first:

GET https://github.com/alice/math-utils/releases/download/SNAPSHOT/math-utils-min.js
GET https://github.com/alice/math-utils/releases/download/SNAPSHOT/math-utils.js

Falls back to main branch raw files:

GET https://raw.githubusercontent.com/alice/math-utils/main/libs/math-utils.js

No integrity verification - SNAPSHOT versions change constantly
Dependency resolution works - Can depend on other SNAPSHOTs

Mixed Development Dependencies

Real-world development often mixes stable releases with local development:

-- Production dependencies
REQUIRE "github.com/alice/math-utils@v1.2.3"     -- Stable release
REQUIRE "github.com/bob/chart-lib@v2.1.0"        -- Stable release

-- Development dependencies  
REQUIRE "./my-new-feature.js"                     -- Local development
REQUIRE "github.com/carol/experimental@SNAPSHOT"  -- Latest experimental
REQUIRE "../shared/team-utils.js"                 -- Shared team library

-- All work together seamlessly
LET result = CALCULATE_STATISTICS data=data       -- From stable release
LET enhanced = MY_NEW_FEATURE data=result         -- From local dev
LET chart = CREATE_CHART data=enhanced            -- From stable release

Development Best Practices

Local file organization:

project/
├── main.rexx                    ← Main script
├── libs/
│   ├── my-feature.js           ← REQUIRE "./libs/my-feature.js"
│   └── helpers.js              ← REQUIRE "./libs/helpers.js"  
├── shared/
│   └── common.js               ← REQUIRE "./shared/common.js"
└── tests/
    └── test-runner.rexx

Dependency declarations in local files:

/*!
 * @rexxjs-meta {
 *   "dependencies": {
 *     "github.com/lodash/lodash": "latest",  // External stable
 *     "./helpers.js": "dev",                // Local relative
 *     "../shared/common.js": "dev"          // Local relative (parent)  
 *   }
 * }
 */

Transitioning to releases:

Development: REQUIRE "./my-lib.js"
Pre-release: REQUIRE "github.com/me/my-lib@SNAPSHOT"
Release: REQUIRE "github.com/me/my-lib@v1.0.0"

Zero-Overhead Detection

Before making network requests, REQUIRE checks if library is already loaded:

// Check for library detection function
const detectionFunc = `${libraryName.toUpperCase().replace(/[^A-Z0-9]/g, '_')}_MAIN`;

// Try to find function in global scope
if (typeof window !== 'undefined' && window[detectionFunc]) {
  // Already loaded - no network request needed
  return true;
}

Transitive Dependency Resolution

@rexxjs-meta Format Specification

The @rexxjs-meta annotation contains JSON metadata about the library:

interface RexxJSMeta {
  // Runtime dependencies (the only type that matters for REQUIRE)
  dependencies?: { [moduleName: string]: string };   // Libraries to load before this one
  
  // Library information (optional)
  name?: string;           // Human-readable library name
  version?: string;        // Semantic version (e.g., "1.2.3")
  author?: string;         // Library author/organization
  license?: string;        // License identifier (e.g., "MIT", "Apache-2.0")
  homepage?: string;       // Documentation/project URL
  repository?: string;     // Source code repository URL
  
  // Build metadata (optional)
  minified?: boolean;      // Whether this file is minified
  buildTime?: string;      // ISO timestamp of build
  buildTool?: string;      // Build tool used (webpack, rollup, etc.)
  
  // Security metadata (optional)
  engines?: { [engine: string]: string };  // Required engine versions
}

Example with full metadata:

/*!
 * advanced-analytics v2.1.0 | (c) 2024 DataCorp | MIT License
 * 
 * @rexxjs-meta {
 *   "dependencies": {
 *     "github.com/alice/math-utils": "^1.0.0",
 *     "github.com/bob/chart-lib": "latest"
 *   },
 *   "name": "Advanced Analytics Library",
 *   "version": "2.1.0",
 *   "author": "DataCorp Engineering",
 *   "license": "MIT",
 *   "homepage": "https://datacorp.com/analytics",
 *   "repository": "https://github.com/datacorp/advanced-analytics",
 *   "minified": true,
 *   "buildTime": "2024-01-15T10:30:00Z",
 *   "buildTool": "webpack@5.89.0",
 *   "engines": {
 *     "rexxjs": ">=1.0.0",
 *     "node": ">=14.0.0"
 *   }
 * }
 */

Dependency Declaration Methods

Libraries can declare dependencies using multiple approaches:

1. Important Comments (Minification-Safe) - RECOMMENDED:

/*!
 * awesome-data-lib v2.1.0 | MIT License
 * 
 * Dependencies:
 * @rexxjs-meta {
 *   "dependencies": {
 *     "github.com/alice/math-utils": "^1.0.0",
 *     "github.com/bob/chart-lib": "latest"
 *   }
 * }
 */

2. Runtime Metadata (Always Works):

const dataAnalysis = {
  'DATA_ANALYSIS_MAIN': () => ({
    type: 'library_info',
    dependencies: [
      'github.com/alice/math-utils@^1.0.0',
      'github.com/bob/chart-lib@latest'
    ],
    loaded: true
  }),
  // ... library functions
};

3. Standard JSON Format (Development Only):

/**
 * @rexxjs-meta-start
 * {
 *   "dependencies": {
 *     "github.com/alice/math-utils": "^1.0.0"
 *   }
 * }
 * @rexxjs-meta-end
 */

Dependency Resolution Algorithm

Load primary library
Extract dependencies from library code (using priority order above)
Recursively load dependencies using same REQUIRE process
Track dependency graph to prevent circular dependencies
Register all functions after entire dependency tree is loaded

Example:

REQUIRE "github.com/alice/data-analysis"

-- This automatically loads:
-- 1. github.com/alice/data-analysis
-- 2. github.com/alice/math-utils (dependency)
-- 3. github.com/bob/chart-lib (dependency)
-- 4. Any dependencies of those libraries

Circular Dependency Detection

// Maintains loading queue to detect cycles
if (this.loadingQueue.has(libraryName)) {
  throw new Error(`Circular dependency detected: ${libraryName} is already loading`);
}

Library Publishing Guide

GitHub Releases Convention

Required naming for library authors:

Release v1.2.3:
├── <libraryName>-min.js    ← PREFERRED (tried first)
├── <libraryName>.js        ← REQUIRED (tried second)
└── CHANGELOG.md

Examples:

math-utils-min.js and math-utils.js
chart-lib-min.js and chart-lib.js

Library Structure Template

/*!
 * my-library v1.0.0 | MIT License
 * @rexxjs-meta {"dependencies":{"lodash":"^4.17.0"}}
 */

const myLibrary = {
  // Primary detection function (REQUIRED)
  'MY_LIBRARY_MAIN': () => ({
    type: 'library_info',
    name: 'My Library',
    version: '1.0.0',
    dependencies: ['lodash@^4.17.0'], // Backup metadata
    loaded: true
  }),

  // Your functions
  'DO_SOMETHING': (param) => {
    // Implementation here
  }
};

// Dual environment export
if (typeof module !== 'undefined' && module.exports) {
  // Node.js
  module.exports = { 'my-library': myLibrary };
  if (typeof global !== 'undefined') {
    global['my-library'] = myLibrary;
  }
} else if (typeof window !== 'undefined') {
  // Browser
  window['my-library'] = myLibrary;
}

Operations and Functions in Libraries

RexxJS libraries can export both operations (imperative commands) and functions (query/expression calls), enabling a clean separation between state-changing actions and data retrieval.

Library Structure with Operations and Functions

/*!
 * bathhouse-library v1.0.0 | MIT License
 * @rexxjs-meta=BATHHOUSE_FUNCTIONS_MAIN
 */

// Metadata provider function
function BATHHOUSE_FUNCTIONS_META() {
  return {
    canonical: "org.rexxjs.examples/bathhouse",
    type: "functions-library",
    name: 'Bathhouse Functions',
    version: '1.0.0',
    description: 'Example library with operations and functions',
    functions: {
      'GUEST_STATUS': { description: 'Get guest status', params: ['guest'] },
      'COUNT_TOKENS': { description: 'Count issued tokens', params: [] },
      'IDENTIFY_SPIRIT': { description: 'Identify spirit', params: ['description'] }
    },
    operations: {
      'SERVE_GUEST': { description: 'Serve a guest', params: ['guest', 'bath'] },
      'CLEAN_BATHHOUSE': { description: 'Clean area', params: ['area', 'intensity'] },
      'ISSUE_TOKEN': { description: 'Issue work token', params: ['worker', 'task'] }
    },
    detectionFunction: 'BATHHOUSE_FUNCTIONS_MAIN'
  };
}

// State for operations to modify
const bathhouse = {
  guests: new Map(),
  tokens: [],
  log: []
};

// Operations: Side-effect commands (receive params object)
const bathhouseOperations = {
  'SERVE_GUEST': function(params) {
    const { guest, bath = 'regular' } = params;
    bathhouse.guests.set(guest, { bath, served: true });
    bathhouse.log.push(`Served ${guest} in ${bath} bath`);
    return { success: true };
  },

  'CLEAN_BATHHOUSE': function(params) {
    const { area = 'main_hall', intensity = 'normal' } = params;
    bathhouse.log.push(`Cleaned ${area} (${intensity})`);
    return { success: true };
  },

  'ISSUE_TOKEN': function(params) {
    const { worker, task = 'cleaning' } = params;
    bathhouse.tokens.push({ worker, task, issued: Date.now() });
    bathhouse.log.push(`Issued token to ${worker}`);
    return { success: true };
  }
};

// Functions: Query operations (receive positional args)
const bathhouseFunctions = {
  'GUEST_STATUS': function(guest) {
    const guestData = bathhouse.guests.get(guest);
    return guestData ? 'satisfied' : 'not_found';
  },

  'COUNT_TOKENS': function() {
    return bathhouse.tokens.length;
  },

  'IDENTIFY_SPIRIT': function(description) {
    const spirits = {
      'muddy': 'river_spirit',
      'hungry': 'no_face',
      'quiet': 'radish_spirit'
    };
    return spirits[description] || 'unknown_spirit';
  }
};

// Combine all exports
const bathhouseFunctionsAll = {
  // Detection function (required by REQUIRE system)
  'BATHHOUSE_FUNCTIONS_MAIN': () => BATHHOUSE_FUNCTIONS_META(),

  // Functions (return values)
  ...bathhouseFunctions,

  // Operations (side effects)
  ...bathhouseOperations
};

// Dual environment export
if (typeof module !== 'undefined' && module.exports) {
  module.exports = bathhouseFunctionsAll;
} else if (typeof window !== 'undefined') {
  Object.assign(window, bathhouseFunctionsAll);
}

Using Operations and Functions

-- Load library with both operations and functions
REQUIRE "cwd:libs/bathhouse-library.js"

-- Operations: Imperative commands (no parentheses)
SERVE_GUEST guest="river_spirit" bath="herbal"
CLEAN_BATHHOUSE area="main_hall" intensity="deep"
ISSUE_TOKEN worker="chihiro" task="cleaning"

-- Functions: Query operations (with parentheses)
LET capacity = BATHHOUSE_CAPACITY()
LET spirit = IDENTIFY_SPIRIT(description="muddy")
LET count = COUNT_TOKENS()

-- Functions support both positional and named parameters
LET spirit1 = IDENTIFY_SPIRIT("hungry")            -- Positional
LET spirit2 = IDENTIFY_SPIRIT(description="quiet") -- Named

-- Named parameters work in pipe operator
LET result = "  hello  " |> STRIP() |> SUBSTR(start=2, length=3)

Using REQUIRE AS with Operations and Functions

-- Prefix both operations and functions
REQUIRE "cwd:libs/bathhouse-library.js" AS bh_(.*)

-- Operations with prefix
bh_SERVE_GUEST guest="no_face" bath="luxury"
bh_CLEAN_BATHHOUSE area="lobby"

-- Functions with prefix
LET capacity = bh_BATHHOUSE_CAPACITY()
LET log = bh_GET_LOG()
LET spirit = bh_IDENTIFY_SPIRIT(description="hungry")

Key Differences

Feature	Operations	Functions
Call Syntax	No parentheses	Always use parentheses
Parameters	Named only (object)	Positional OR named
Purpose	Side effects, state changes	Return values, queries
In Expressions	❌ Not allowed	✅ Works everywhere
In Pipes	❌ Not allowed	✅ Full support
Registration	Via metadata `operations`	Via metadata `functions`

Parameter Handling

Operations receive params object:

'SERVE_GUEST': function(params) {
  const { guest, bath = 'regular' } = params;
  // params is the raw named parameters object
}

Functions receive positional args:

'IDENTIFY_SPIRIT': function(description) {
  // description is the first positional arg
  // parameter-converter handles named → positional conversion
}

When called with named params IDENTIFY_SPIRIT(description="muddy"), the parameter-converter transforms it to IDENTIFY_SPIRIT("muddy") before calling the function.

Build Script Example

{
  "scripts": {
    "build": "webpack --mode=development --output-filename=my-library.js",
    "build:min": "webpack --mode=production --output-filename=my-library-min.js", 
    "build:release": "npm run build && npm run build:min",
    "release": "gh release create v$npm_package_version dist/my-library.js dist/my-library-min.js"
  }
}

GitHub Actions Release Automation

name: Release
on:
  push:
    tags: ['v*']

jobs:
  release:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v3
      - uses: actions/setup-node@v3
      
      - name: Build library files with correct naming
        run: |
          npm ci
          npm run build:release
        
      - name: Create GitHub release with standard naming
        uses: softprops/action-gh-release@v1
        with:
          files: |
            dist/$.js
            dist/$-min.js
        env:
          GITHUB_TOKEN: $

Minification-Safe Dependencies

The Problem

Standard comments are removed during minification:

/* @dependencies github.com/alice/math-utils */  // ← Gone after minification!

The Solution: Important Comments

Use /*! comments that all minifiers preserve by default:

/*!
 * my-library v1.0.0 | MIT License
 * @rexxjs-meta {"dependencies":{"github.com/alice/math-utils":"^1.0.0"}}
 */

Key benefits:

✅ Zero configuration - Works with default minifier settings
✅ Perfect preservation - Newlines and indentation intact
✅ Universal support - Webpack, Rollup, ESBuild, Terser all preserve /*!
✅ Established pattern - Same approach as jQuery, Lodash, Bootstrap

Default Minifier Behavior

These commands preserve important comments out-of-the-box:

webpack --mode=production
rollup -c
esbuild --minify src/lib.js
terser src/lib.js

Before and After Minification

Before (source):

/*!
 * math-utils v1.0.0 | MIT License
 * @rexxjs-meta {"dependencies":{"lodash":"4.17.21"}}
 */
const mathUtils = {
  'MATH_UTILS_MAIN': () => ({ loaded: true }),
  'ADD': (a, b) => a + b
};
if (typeof window !== 'undefined') window['math-utils'] = mathUtils;

After (minified):

/*!
 * math-utils v1.0.0 | MIT License  
 * @rexxjs-meta {"dependencies":{"lodash":"4.17.21"}}
 */
const a={MATH_UTILS_MAIN:()=>({loaded:!0}),ADD:(a,b)=>a+b};"undefined"!=typeof window&&(window["math-utils"]=a);

✅ Perfect preservation! Dependencies survive minification automatically.

Testing Your Library

-- Test basic loading
REQUIRE "github.com/yourname/your-library@v1.2.3"

-- Test detection function
LET info = YOUR_LIBRARY_MAIN
SAY "Library:" info.name
SAY "Version:" info.version

-- Test main functionality
LET result = YOUR_MAIN_FUNCTION param="test"
SAY "Result:" result

Error Handling

-- REQUIRE throws errors on failure
SIGNAL ON ERROR

REQUIRE "github.com/nonexistent/library@v1.0.0"

-- This won't execute if library fails to load
SAY "Library loaded successfully"
EXIT

ERROR:
SAY "Failed to load library:" ERRORTEXT()

Security Considerations

Libraries execute with full access to global scope
Only load libraries from trusted sources
Dependency metadata is parsed from untrusted code
Consider implementing sandboxing for production use

Migration from ZIP Releases

Before (❌):

Release v1.2.3:
└── dist.zip
    ├── lib/my-library.js
    └── lib/my-library.min.js

After (✅):

Release v1.2.3:
├── my-library.js          ← Direct download
├── my-library-min.js      ← Direct download  
└── CHANGELOG.md

Migration script:

# Extract from existing ZIP and republish
unzip dist.zip
gh release upload v1.2.3 lib/my-library.js lib/my-library-min.js

Best Practices

For Library Authors

Always publish both development and minified versions
Use important comments (/*!) for dependency metadata
Include runtime metadata in detection function (double redundancy)
Test minified versions preserve dependency information
Follow GitHub releases naming convention

For Library Users

Prefer versioned imports over @latest
Test transitive dependency loading
Monitor network requests in development tools
Cache libraries in production environments

Remote REQUIRE via CHECKPOINT

SCRO (Source-Controlled Remote Orchestration)

The Remote REQUIRE system enables RexxJS scripts to load libraries from remote sources through a bidirectional communication channel using the CHECKPOINT protocol. This advanced feature supports distributed computing scenarios where the RexxJS interpreter runs in a controlled environment and needs to request libraries from an orchestration service.

Environment Detection

Remote orchestration is detected through several mechanisms:

Environment Variables:

export SCRO_REMOTE=true
export SCRO_ORCHESTRATION_ID=orch_123456

RexxJS Variables:

-- Set remoteness via interpreter variables
LET SCRO_REMOTE = "true"
LET SCRO_ORCHESTRATION_ID = "session_789"

Streaming Callback Context:

// Orchestration environment detected via streaming callback presence
interpreter.streamingProgressCallback = (message) => {
  // Handle CHECKPOINT messages
};

CHECKPOINT Communication Protocol

The CHECKPOINT system uses a JSON-based messaging protocol for bidirectional communication:

Request Message Format:

{
  type: 'rexx-require',
  subtype: 'require_request', 
  data: {
    type: 'require_request',
    libraryName: 'my-remote-library',
    requireId: 'req_12345',
    timestamp: 1640995200000
  }
}

Response Message Format:

{
  type: 'rexx-require-response',
  requireId: 'req_12345',
  success: true,
  libraryCode: 'module.exports = { ... };',
  libraryName: 'my-remote-library'
}

Built-in vs Third-party Detection

The remote REQUIRE system intelligently routes library requests:

Built-in Libraries (Local Loading):

-- These load locally even in remote context
REQUIRE "string-functions"    -- Built-in, loads locally
REQUIRE "math-functions"      -- Built-in, loads locally
REQUIRE "./src/local-lib.js"  -- src/ directory, loads locally

Third-party Libraries (Remote Loading):

-- These request via CHECKPOINT in remote context
REQUIRE "custom-library"              -- Remote request
REQUIRE "github-user/repo"            -- Remote request  
REQUIRE "./tests/test-lib.js"         -- Non-src local file, remote request

Remote Library Execution

When a library is received via CHECKPOINT, it’s executed safely in the interpreter’s context:

// Remote library code execution
await interpreter.executeRemoteLibraryCode(libraryName, libraryCode);

// Functions become available immediately
const result = interpreter.functions.REMOTE_FUNCTION('param');

Library Caching:

// Libraries are cached after loading
const cached = interpreter.libraryCache.get('remote-library');
// { loaded: true, code: '...', timestamp: 1640995200000 }

Security Considerations

Code Execution:

Remote library code executes with full interpreter privileges
Libraries run in the same context as the main script
No sandboxing or permission restrictions applied

Network Security:

All communication goes through the CHECKPOINT channel
No direct network requests from remote-orchestrated interpreters
Library resolution handled by orchestration service

Trust Model:

Remote libraries must be trusted by the orchestration service
Library code integrity depends on the orchestration environment
Consider implementing additional validation for production use

Error Handling

Timeout Handling:

// Configurable timeout for remote requests
const response = await interpreter.waitForCheckpointResponse(requireId, 5000);

if (response.success === false && response.error === 'timeout') {
  throw new Error(`Remote REQUIRE timeout for ${libraryName}`);
}

Communication Channel Errors:

// Handle missing communication channel
if (!interpreter.streamingProgressCallback && !window?.parent?.postMessage) {
  return { success: false, error: 'no_communication_channel' };
}

Library Resolution Errors:

SIGNAL ON ERROR

REQUIRE "non-existent-remote-library"

ERROR:
  SAY "Remote REQUIRE failed:" ERRORTEXT()
  -- Error: Remote REQUIRE failed for non-existent-remote-library: Library not found

Implementation Example

Setting up Remote Context:

-- Configure for remote orchestration
LET SCRO_REMOTE = "true"
LET SCRO_ORCHESTRATION_ID = "demo_session_123"

-- Load remote library
REQUIRE "analytics-package"

-- Use loaded functions
LET data = PROCESS_ANALYTICS input=rawData
LET report = GENERATE_REPORT data=data

Orchestration Service Integration:

// Orchestration service handles CHECKPOINT messages
interpreter.streamingProgressCallback = async (message) => {
  if (message.type === 'rexx-require') {
    const { libraryName, requireId } = message.data;
    
    // Resolve library from registry/cache
    const libraryCode = await resolveLibrary(libraryName);
    
    // Send response
    window.postMessage({
      type: 'rexx-require-response', 
      requireId: requireId,
      success: true,
      libraryCode: libraryCode,
      libraryName: libraryName
    });
  }
};

Development Workflow

Phase 1: Local Development

-- Normal local development (no remote context)
REQUIRE "github.com/user/library@latest"  -- Direct GitHub loading

Phase 2: Remote Testing

# Set up remote orchestration environment
export SCRO_REMOTE=true
export SCRO_ORCHESTRATION_ID=test_123

# Run with orchestration service
node orchestrator.js --script=my-script.rexx

Phase 3: Production Deployment

-- Script automatically detects remote context
-- No code changes needed - REQUIRE routing is automatic
REQUIRE "production-library"  -- Loaded via orchestration service

Performance Characteristics

Local Library Loading:

✅ Zero latency - Built-in libraries load immediately
✅ No network overhead - Direct function registration
✅ Caching efficient - Functions stored in memory

Remote Library Loading:

⚠️ Network latency - CHECKPOINT round-trip required
⚠️ Timeout risk - Communication channel dependent
✅ One-time cost - Libraries cached after first load

Use Cases

Controlled Execution Environments:

Jupyter notebook integrations
Sandboxed code execution platforms
Enterprise policy-controlled environments

Distributed Computing:

Microservice architectures
Container orchestration platforms
Serverless function environments

Security-Conscious Deployments:

Library approval workflows
Centralized dependency management
Audit trail requirements

INTERPRET - Dynamic code execution
JSON Functions - Parse dependency metadata
Web Functions - HTTP resource access
Security Functions - Cryptographic verification

Total Libraries Loaded: Dynamic based on requirements Environments: Browser (fetch), Node.js (https.get), and Remote Orchestration (CHECKPOINT) Security: Transitive dependency validation, circular dependency detection, and controlled remote execution

REQUIRE System - Dynamic Library Loading

Basic Syntax

Environment Detection and Loading

Browser Environment

Node.js Environment

Library Loading Sequence

Fallback GET Sequence

Development Workflows

Local File References

Node.js Development Mode (Command Line)

Browser Development Mode (Webpack Required)

Development Environment Comparison

Local Library Development Workflow

SNAPSHOT Versions

Mixed Development Dependencies

Development Best Practices

Zero-Overhead Detection

Transitive Dependency Resolution

@rexxjs-meta Format Specification

Dependency Declaration Methods

Dependency Resolution Algorithm

Circular Dependency Detection

Library Publishing Guide

GitHub Releases Convention

Library Structure Template

Operations and Functions in Libraries

Library Structure with Operations and Functions

Using Operations and Functions

Using REQUIRE AS with Operations and Functions

Key Differences

Parameter Handling

Build Script Example

GitHub Actions Release Automation

Minification-Safe Dependencies

The Problem

The Solution: Important Comments

Default Minifier Behavior

Before and After Minification

Testing Your Library

Error Handling

Security Considerations

Migration from ZIP Releases

Best Practices

For Library Authors

For Library Users

Remote REQUIRE via CHECKPOINT

SCRO (Source-Controlled Remote Orchestration)

Environment Detection

CHECKPOINT Communication Protocol

Built-in vs Third-party Detection

Remote Library Execution

Security Considerations

Error Handling

Implementation Example

Development Workflow

Performance Characteristics

Use Cases

Related Functions