Feature implementation from commits caf17ec..c9e752c #2

yashuatla · 2025-06-23T17:38:50Z

This PR contains changes from a range of commits from the original repository.

Commit Range: caf17ec..c9e752c
Files Changed: 7 (4 programming files)
Programming Ratio: 57.1%

Commits included:

Fix build error with GCC by forcing Clang compiler in CMake on android/aarch64 (Fix build error with GCC by forcing Clang in CMake on android/aarch64 microsoft/BitNet#242)
Add run_inference_server.py for Running llama.cpp Built-in Server (Add run_inference_server.py for Running llama.cpp Built-in Server microsoft/BitNet#204)
Merge pull request Fix model architecture name microsoft/BitNet#212 from microsoft/arch-name-dev
Fix model architecture name
Merge pull request refine readme microsoft/BitNet#176 from microsoft/readme-dev
refine readme
Merge pull request add third-party demo microsoft/BitNet#175 from microsoft/readme-dev
add third-party demo
Update README.md
Update README.md (Update README.md microsoft/BitNet#172)
... and 5 more commits

add support for bitnet2b_2501 model

…microsoft#171) * update readme and setup file for new model. * update model file name --------- Co-authored-by: Yan Xia <[email protected]>

add two FAQs for windows build requestions.

add third-party demo

refine readme

Fix model architecture name

…crosoft#204) * Update CMakeLists.txt I added a CMake option to compile the Llama.cpp server. This update allows us to easily build and deploy the server using BitNet * Create run_inference_server.py same as run_inference, but for use with llama.cpp's built in server, for some extra comfort In particular: - The build directory is determined based on whether the system is running on Windows or not. - A list of arguments (`--model`, `-m` etc.) is created. - The main argument list is parsed and passed to the `subprocess.run()` method to execute the system command.

…d/aarch64 (microsoft#242) GCC does not recognize Clang-specific warning flags like -Wunreachable-code-break and -Wunreachable-code-return, which are passed by upstream submodules (e.g., ggml). This patch forces CMake to use Clang via command-line arguments, avoiding the need to patch nested submodules. This resolves compiler errors without modifying submodule source code.

codeowlai · 2025-06-23T18:29:50Z

setup_env.py

        exit(0)
    logging.info("Compiling the code using CMake.")
-    run_command(["cmake", "-B", "build", *COMPILER_EXTRA_ARGS[arch], *OS_EXTRA_ARGS.get(platform.system(), [])], log_step="generate_build_files")
+    run_command(["cmake", "-B", "build", *COMPILER_EXTRA_ARGS[arch], *OS_EXTRA_ARGS.get(platform.system(), []), "-DCMAKE_C_COMPILER=clang", "-DCMAKE_CXX_COMPILER=clang++"], log_step="generate_build_files")


🐛 Correctness Issue

Hardcoded compiler breaks cross-platform compatibility.

Hardcoding clang/clang++ will cause build failures on systems where these compilers aren't available or where different compilers are required.

Current Code (Diff):

- run_command(["cmake", "-B", "build", *COMPILER_EXTRA_ARGS[arch], *OS_EXTRA_ARGS.get(platform.system(), []), "-DCMAKE_C_COMPILER=clang", "-DCMAKE_CXX_COMPILER=clang++"], log_step="generate_build_files") + run_command(["cmake", "-B", "build", *COMPILER_EXTRA_ARGS[arch], *OS_EXTRA_ARGS.get(platform.system(), [])], log_step="generate_build_files")

📝 Committable suggestion

‼️ IMPORTANT
Trust, but verify! 🕵️ Please review this suggestion with the care of a code archaeologist - check that it perfectly replaces the highlighted code, preserves all lines, maintains proper indentation, and won't break anything in production. Your future self will thank you! 🚀

Suggested change

run_command(["cmake", "-B", "build", *COMPILER_EXTRA_ARGS[arch], *OS_EXTRA_ARGS.get(platform.system(), []), "-DCMAKE_C_COMPILER=clang", "-DCMAKE_CXX_COMPILER=clang++"], log_step="generate_build_files")

run_command(["cmake", "-B", "build", *COMPILER_EXTRA_ARGS[arch], *OS_EXTRA_ARGS.get(platform.system(), [])], log_step="generate_build_files")

codeowlai · 2025-06-23T18:29:54Z

utils/convert-ms-to-gguf-bitnet.py

+) -> Tuple[np.ndarray, np.ndarray]:
+    M, K = w.shape
+
+    cf=configparser.ConfigParser()


🐛 Correctness Issue

Hardcoded path with no error handling.

The code reads a configuration file from a hardcoded path without checking if the file exists, which will cause runtime errors if the file is missing.

Current Code (Diff):

- cf=configparser.ConfigParser() - cf.read("./build/kcfg.ini") + cf = configparser.ConfigParser() + try: + if not os.path.exists("./build/kcfg.ini"): + raise FileNotFoundError("Configuration file not found") + cf.read("./build/kcfg.ini") + if len(cf.sections()) == 0: + raise ValueError("Configuration file is empty or invalid") + except (FileNotFoundError, ValueError) as e: + logger.error(f"Error loading configuration: {e}") + raise

📝 Committable suggestion

‼️ IMPORTANT
Trust, but verify! 🕵️ Please review this suggestion with the care of a code archaeologist - check that it perfectly replaces the highlighted code, preserves all lines, maintains proper indentation, and won't break anything in production. Your future self will thank you! 🚀

Suggested change

cf=configparser.ConfigParser()

cf = configparser.ConfigParser()

try:

if not os.path.exists("./build/kcfg.ini"):

raise FileNotFoundError("Configuration file not found")

cf.read("./build/kcfg.ini")

if len(cf.sections()) == 0:

raise ValueError("Configuration file is empty or invalid")

except (FileNotFoundError, ValueError) as e:

logger.error(f"Error loading configuration: {e}")

raise

codeowlai · 2025-06-23T18:29:57Z

utils/convert-ms-to-gguf-bitnet.py

+
+    return w
+
+def transform_to_i2(x : NDArray):


🐛 Correctness Issue

Potential division by zero.

The transform_to_i2 function doesn't properly handle the case where all values in tile_x are zero, which could lead to unexpected behavior.

Current Code (Diff):

-def transform_to_i2(x : NDArray): - x_num = np.prod(x.shape) - tile_x = np.reshape(x, x_num) - scale = 1 - for i in range(x_num): - if tile_x[i] != 0: - scale = tile_x[i] - break - tile_x = np.divide(tile_x, scale) - tile_x = (tile_x.astype(np.int8) + 2).astype(np.uint8) - ans = np.reshape(tile_x, x.shape) - return ans, scale +def transform_to_i2(x : NDArray): + x_num = np.prod(x.shape) + tile_x = np.reshape(x, x_num) + scale = 1 + found_nonzero = False + for i in range(x_num): + if tile_x[i] != 0: + scale = tile_x[i] + found_nonzero = True + break + + if not found_nonzero: + # Handle the all-zeros case + return np.zeros(x.shape, dtype=np.uint8), scale + + tile_x = np.divide(tile_x, scale) + tile_x = (tile_x.astype(np.int8) + 2).astype(np.uint8) + ans = np.reshape(tile_x, x.shape) + return ans, scale

📝 Committable suggestion

‼️ IMPORTANT
Trust, but verify! 🕵️ Please review this suggestion with the care of a code archaeologist - check that it perfectly replaces the highlighted code, preserves all lines, maintains proper indentation, and won't break anything in production. Your future self will thank you! 🚀

Suggested change

def transform_to_i2(x : NDArray):

def transform_to_i2(x : NDArray):

x_num = np.prod(x.shape)

tile_x = np.reshape(x, x_num)

scale = 1

found_nonzero = False

for i in range(x_num):

if tile_x[i] != 0:

scale = tile_x[i]

found_nonzero = True

break

if not found_nonzero:

# Handle the all-zeros case

return np.zeros(x.shape, dtype=np.uint8), scale

tile_x = np.divide(tile_x, scale)

tile_x = (tile_x.astype(np.int8) + 2).astype(np.uint8)

ans = np.reshape(tile_x, x.shape)

return ans, scale

potassiummmm and others added 15 commits March 12, 2025 18:16

add support for bitnet2b_2501 model

4f2e41a

add conversion logic for new model

09f9106

Merge pull request microsoft#167 from potassiummmm/bitnet-25

fa854cf

add support for bitnet2b_2501 model

update readme and setup script to support official BitNet b1.58 model (…

fd3f355

…microsoft#171) * update readme and setup file for new model. * update model file name --------- Co-authored-by: Yan Xia <[email protected]>

Update README.md

0e7dadb

Update README.md (microsoft#172)

8f75f99

add two FAQs for windows build requestions.

Update README.md

1c77bd8

add third-party demo

71fdd94

Merge pull request microsoft#175 from microsoft/readme-dev

034b34c

add third-party demo

refine readme

874e6bd

Merge pull request microsoft#176 from microsoft/readme-dev

fd9f1d6

refine readme

Fix model architecture name

488dc1e

Merge pull request microsoft#212 from microsoft/arch-name-dev

c17d1c5

Fix model architecture name

codeowlai bot reviewed Jun 23, 2025

View reviewed changes

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Feature implementation from commits caf17ec..c9e752c #2

Feature implementation from commits caf17ec..c9e752c #2

Uh oh!

yashuatla commented Jun 23, 2025

Uh oh!

codeowlai bot Jun 23, 2025

Uh oh!

codeowlai bot Jun 23, 2025

Uh oh!

codeowlai bot Jun 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

	run_command(["cmake", "-B", "build", COMPILER_EXTRA_ARGS[arch], OS_EXTRA_ARGS.get(platform.system(), []), "-DCMAKE_C_COMPILER=clang", "-DCMAKE_CXX_COMPILER=clang++"], log_step="generate_build_files")
	run_command(["cmake", "-B", "build", COMPILER_EXTRA_ARGS[arch], OS_EXTRA_ARGS.get(platform.system(), [])], log_step="generate_build_files")

-    cf=configparser.ConfigParser()
+    cf = configparser.ConfigParser()
+    try:
+        if not os.path.exists("./build/kcfg.ini"):
+            raise FileNotFoundError("Configuration file not found")
+        cf.read("./build/kcfg.ini")
+        if len(cf.sections()) == 0:
+            raise ValueError("Configuration file is empty or invalid")
+    except (FileNotFoundError, ValueError) as e:
+        logger.error(f"Error loading configuration: {e}")
+        raise

-def transform_to_i2(x : NDArray):
+def transform_to_i2(x : NDArray):
+    x_num = np.prod(x.shape)
+    tile_x = np.reshape(x, x_num)
+    scale = 1
+    found_nonzero = False
+    for i in range(x_num):
+        if tile_x[i] != 0:
+            scale = tile_x[i]
+            found_nonzero = True
+            break
+    if not found_nonzero:
+        # Handle the all-zeros case
+        return np.zeros(x.shape, dtype=np.uint8), scale
+    tile_x = np.divide(tile_x, scale)
+    tile_x = (tile_x.astype(np.int8) + 2).astype(np.uint8)
+    ans = np.reshape(tile_x, x.shape)
+    return ans, scale

Feature implementation from commits caf17ec..c9e752c #2

Are you sure you want to change the base?

Feature implementation from commits caf17ec..c9e752c #2

Uh oh!

Conversation

yashuatla commented Jun 23, 2025

Uh oh!

codeowlai bot Jun 23, 2025

Choose a reason for hiding this comment

Uh oh!

codeowlai bot Jun 23, 2025

Choose a reason for hiding this comment

Uh oh!

codeowlai bot Jun 23, 2025

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants