unproject function

bool unproject (Matrix4 cameraMatrix, num viewportX, num viewportWidth, num viewportY, num viewportHeight, num pickX, num pickY, num pickZ, Vector3 pickWorld)

On success, Sets pickWorld to be the world space position of the screen space pickX, pickY, and pickZ.

The viewport is specified by (viewportX, viewportWidth) and (viewportY, viewportHeight).

cameraMatrix includes both the projection and view transforms.

pickZ is typically either 0.0 (near plane) or 1.0 (far plane).

Returns false on error, for example, the mouse is not in the viewport

Implementation 

bool unproject(
    Matrix4 cameraMatrix,
    num viewportX,
    num viewportWidth,
    num viewportY,
    num viewportHeight,
    num pickX,
    num pickY,
    num pickZ,
    Vector3 pickWorld) {
  viewportX = viewportX.toDouble();
  viewportWidth = viewportWidth.toDouble();
  viewportY = viewportY.toDouble();
  viewportHeight = viewportHeight.toDouble();
  pickX = pickX.toDouble();
  pickY = pickY.toDouble();
  pickX = (pickX - viewportX);
  pickY = (pickY - viewportY);
  pickX = (2.0 * pickX / viewportWidth) - 1.0;
  pickY = (2.0 * pickY / viewportHeight) - 1.0;
  pickZ = (2.0 * pickZ) - 1.0;

  // Check if pick point is inside unit cube
  if (pickX < -1.0 ||
      pickY < -1.0 ||
      pickX > 1.0 ||
      pickY > 1.0 ||
      pickZ < -1.0 ||
      pickZ > 1.0) {
    return false;
  }

  // Copy camera matrix.
  final Matrix4 invertedCameraMatrix = new Matrix4.copy(cameraMatrix);
  // Invert the camera matrix.
  invertedCameraMatrix.invert();
  // Determine intersection point.
  final Vector4 v =
      new Vector4(pickX.toDouble(), pickY.toDouble(), pickZ.toDouble(), 1.0);
  invertedCameraMatrix.transform(v);
  if (v.w == 0.0) {
    return false;
  }
  final double invW = 1.0 / v.w;
  pickWorld
    ..x = v.x * invW
    ..y = v.y * invW
    ..z = v.z * invW;

  return true;
}