Execution Time for iOS

NSDate *methodStart = [NSDate date];
// code...
NSDate *methodFinish = [NSDate date];
NSTimeInterval executionTime = [methodFinish timeIntervalSinceDate:methodStart];
NSLog(@"executionTime = %f", executionTime);

Qualcomm Vuforia in Xcode 6

Qualcomm Vuforia 真不是普通的難上手,文件寫的跟天書一樣,
照官方文件裝完,應該有很大的機會跑不起來,先確認一下
Build Settings -> Architectures: armv7 armv7s
目前 4.0.5 Beta 還沒支援 arm64,但官方有說之後會補上。

先設定 license key

SampleApplicationSession.mm
QCAR::setInitParameters(mQCARInitFlags,"add your license key");

接著把要辨識的圖形丟上 Target Manager 並下載回來,會有兩個檔 *.dat
*.xml 內容如下,稍微看一下 ImageTarget name 之後會用到。

<ImageTarget name="ARCODE1" size="500.000000 500.000000" />

設定辨識檔

ImageTargetsViewController.m
dataSetStonesAndChips = [self loadObjectTrackerDataSet:@"your xml file"];

官方所提供的範例做法是 viewController -> EAGLView,EAGLView 包含
OpenGL ES Context 可以放 3D 物件,另外也可以再加上自己的 subview 與一般
ios 做法沒什麼差別,要特別提到是 QCAR 會呼叫 renderFrameQCAR
進行辨識,由於 randerFrameQCAR 是在 background thread 執行,
所以當你要更新 UI 時,要用 performSelectorOnMainThread 去處理。
簡單範例如下

- (void)renderFrameQCAR
{
  [self setFramebuffer];
  // Clear colour and depth buffers
  glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
  
  // Render video background and retrieve tracking state
  QCAR::State state = QCAR::Renderer::getInstance().begin();
  QCAR::Renderer::getInstance().drawVideoBackground();

  // ...  
  for (int i = 0; i < state.getNumTrackableResults(); ++i) {
    // Get the trackable
    const QCAR::TrackableResult* result = state.getTrackableResult(i);
    const QCAR::Trackable& trackable = result->getTrackable();
    
    // Choose the index based on the ImageTarget name
    int targetIndex = 0; // "stones"
    if (!strcmp(trackable.getName(), "chips"))
      targetIndex = 1;
    else if (!strcmp(trackable.getName(), "tarmac"))
      targetIndex = 2;
    else if (!strcmp(trackable.getName(), "ARCODE1"))
      targetIndex = 3;
    
    if (targetIndex==3) {
      [self performSelectorOnMainThread:@selector(showMyView) withObject:nil waitUntilDone:NO];
    }
    SampleApplicationUtils::checkGlError("EAGLView renderFrameQCAR");
  }
  
  // ...
  [self presentFramebuffer];
}

Core Image and Graphic context in swift

Graphic context

// size: 200x200, opaque: true, scale: 1(0:screen scale)
UIGraphicsBeginImageContextWithOptions(CGSize(width: 200, height: 200), true, 1)

// drawing commands
CGContextSetRGBFillColor (context, 1, 1, 0, 1);
// ...

// getting an image from it
let image = UIGraphicsGetImageFromCurrentImageContext()
UIGraphicsEndImageContext()

Core image with filter

// create a UIImage
let image = UIImage(named: "c0415")
// create Core Image context
let ciContext = CIContext(options: nil)
// create a CIImage(image data for processing)
let coreImage = CIImage(image: image)
// picking the filter
//let filter = CIFilter(name: "CIPhotoEffectTransfer")
let filter = CIFilter(name: "CIVignetteEffect")
// passing image
filter.setValue(coreImage, forKey: kCIInputImageKey)
// set a custom value for the inputCenter
filter.setValue(CIVector(x: xpos, y: ypos), forKey: kCIInputCenterKey)
// retrieve the processed image
let filteredImageData = filter.valueForKey(kCIOutputImageKey) as CIImage
// returns a Quartz image from the Core Image context
let filteredImageRef = ciContext.createCGImage(filteredImageData, fromRect: filteredImageData.extent())
// this is our final UIImage ready to be displayed
let filteredImage = UIImage(CGImage: filteredImageRef);

drawing text

// select a font
let font = UIFont.boldSystemFontOfSize(44)
let showText:NSString = "hello world"
// setting attr: font name, color...etc.
let attr = [NSFontAttributeName: font, NSForegroundColorAttributeName:UIColor.whiteColor()]
// getting size
let sizeOfText = showText.sizeWithAttributes(attr)

let image = UIImage(named: "c0415")
let rect = CGRectMake(0, 0, image!.size.width, image!.size.height)

UIGraphicsBeginImageContextWithOptions(CGSize(width: rect.size.width, height: rect.size.height), true, 0)

// drawing our image to the graphics context
image?.drawInRect(rect)
// drawing text
showText.drawInRect(CGRectMake(rect.size.width-sizeOfText.width-10, rect.size.height-sizeOfText.height-10, rect.size.width, rect.size.height), withAttributes: attr)

// getting an image from it
let newImage = UIGraphicsGetImageFromCurrentImageContext();
UIGraphicsEndImageContext()

Size Classes in iOS8

  • w:Any h:Any 將共用元件置入,設定在各種不同裝置下,不會變動的條件,像是圖片的比例...等
  • 接下來針對 portrait/landscape 分別設定
    • 設定 Constraints
    • 設定 UILable/UIButton font size
    • 甚至可以設定是否要加入或移除某些特定的 view

Apple 的文件說需要 xCode 6 之後的版本才支援 size classes
且能向下相容 iOS8 之前的版本,
實際測試
在 iOS7 上的 iphone4 是可執行,大致上都正常,但 font size 沒有正確顯示。

Ref

Building a Game with SceneKit

WWDC2014: Building a Game with SceneKit

Managing Assets

// Load two scenes
SCNScene *mainScene = [SCNScene sceneNmaed:@"level.dae"];
SCNScene *characterScene = [SCNScene sceneNamed:@"monkey.dae"];

// Get the monkey
SCNNode *monkey = [characterScene.rootNode childNodeWithName:@"monkey" recursively:YES];

// Add a monkey to the level
[mainScene.rootNode addChildNode:monkey];

// Add another monkey to the level
[mainScene.rootNode addChildeNode:[monkey clone]];

Behaviors

Animating characters

  • skinned-character.da
  • run.dae
  • jump.dae
  • idle.dae
// Load an animation
CAAnimation *anim = [sceneSource entryWithIdentifier:animationName withClass:[CAAnimation class]];

// Play it
[character addAnimation:anim forKey:@"run"];

Moving characters

Collision Detection

// Be notified through delegation
- (void)physicsWorld:(SCNPhysicsWorld *)world
     didBeginContact:(SCNPhysicsContact *)contact;

// Or explicitly perform ray tests
- (NSArray *)rayTestWithSegmentFromPoint:(SCNVector3)origin
                                 toPoint:(SCNVector3)dest
                                 options:(NSDictionary *)options;

// Animating items
[aBanana runAction:
  [SCNAction repeatActionForever:
    [SCNAction rotateByX:0.0 y:2.0 * M_PI z:0.0 duration:2.0]]];

Particles

// Load a particle system
SCNParticleSystem *particleSystem =
  [SCNParticleSystem particleSystemNamed:@"dust.scnp"
                             inDirectory:@"art.scnassets/particles"];

// Attach to a node
[character addparticleSystem:particleSystem];

// Control emission
particleSystem.birthRate = shouldEmit ? aBirthRate : 0;

Visual Improvements

NSString *modifier = @"_geometry.texcoords[0] +=
vec2(
    sin(_geometry.position.z + u_time) * 0.01,
    -0.05 * u_time
);";

lavaNode.geometry.shaderModifiers =
  @{SCNShaderModifierEntryPointGeometry : modifier};

Postprocessing

  • UIWindow
    • SCNView -SKScene(SpriteKey overlays)
      • Score
      • Timer
      • title screen

Performance Optimization

CPU 瓶頸,通常是 draw calls 過多造成。

Flattening,降低 draw calls 次數

  • Flatten directly in 3D tools(recommended)
  • Flatten programmatically
SCNNode *flattenedNode = [node flattenedClone];

GPU 瓶頸

  • Fill Rate limited(要求 GPU render 過多 pixels)
    • 降低 contents scale factor(1x, 2x),縮小圖檔尺寸 512x512->256x256
    • Reduce screen space postprocesses(shadows,depth of field,reflective floor)
  • Fragment shaders limited
    • 儘可能使用 Static Lighting
    • 設置適當 Omni Light
    • Dynamic shadow 可以用 Projected shadow 代替
    • texture size
    • mipmapping(缺點:載入的時間長,會花比較多的記憶體~30%)

Summary: Performance

  • CPU
    • Reduce draw calls by flattening
    • Less physic bodies
    • Less animations
    • Less actions
  • Tiler
    • Levels of detail
    • Split scenes in chunks
  • Renderer or device
    • Simpler materials
    • Less/Simpler lights
    • Smaller textures, mipmapping
    • Downscaled contents size
    • Less postprocess
    • No multisampling

What's new in SceneKit

WWDC 2014: What's new in SceneKit

Loading a 3D Scene

  • COLLADA .dae
  • Alembic .abc(animation)

SceneKit Editor

Loading a DAE

SCNScene *scene = [SCNScene sceneNamed:@"demo.dae"];

SceneKit Assets Catalog

.scnassets folders

Displaying the Scene

// Assign the scene
aSCNView.scene = aScene;

// modifiy a node
aNode.position = SCNVector3Make(0,0,0);
aNode.scale = SCNVector3Make(2,2,2);
aNode.rotation = SCNVector4Make(x,y,z,angle);
aNode.opacity = 0.5;

Animating a Scene

  • Per-frame updates
  • Animations
  • Actions *new*
  • Physics
  • Constraints

Animations

// Begin a transaction
[SCNTransaction begin];
[SCNTransation setAnimationDuration:2.0];

// Change properties
aNode.opacity = 1.0;
aNode.rotation = SCNVector4(0,1,0,M_PI*4);

// Commit
[SCNTransaction commit];

or

// Create a animation
animation = [CABasicAnimation animationWithKeyPath:@"rotation"];

// Configure
animation.duration = 2.0;
animation.toValue = [NSValue valueWithSCNVector4:SCNVector4Make(0,1,0,M_PI*2)];
animation.repeatCount = MAXFLOAT;

// Play the animation
[aNode addAnimation:animation forKey:@"myAnimation"];

Animation Events

Smooth transitions

// 音效 Playing a sound at 60 percent
SCNAnimationEvent *anEvent = [SCNAnimationEvent animationEventWithKeyTime:0.6 block:aSoundBlock];
anAnimation.animationEvents = @[anEvent, anotherEvent];
// 轉換動作(idle->attack->idle)加上淡入淡出
anAnimation.fadeInDuration = 0.3;
anAnimation.fadeOutDuration = 0.3;

Actions

Easy to sequence, group, and repeat

[aNode runAction:[SCNAction repeatActionForever:[SCNAtion rotateByX:0 y:M_PI*2 z:0 duration:5.0]]];
// 如果需要取得移動中物件位置,可以用 presentationNode.position
node.position != node.presentationNode.position

Physics

// Dynamic body: Make a node dynamic
aNode.physicsBody = [SCNPhysicsBody dynamicBody];

// Manipulate with forces
// Apply an impulse
[aNode.phsicsBody appleyForce:aVector3 atPosition:aVector3 impulse:YES];

// Staic bodies: Make a node static
aNode.physicsBody = [SCNPhysicsBody staticBody];

// Kinematic body: Make a node kinematic(不受碰撞/重力影響,但是可以用程式的方式加入速度)
aNode.physicsBody = [SCNPhysicsBody kinematicBody];

// Physics shape: SceneKit 會自動建立,也可以用手動的方式指定,如下:
aNode.physicsBody.physicsShape = [SCNPhysicsShape shapeWithGeometry:aGeometry options:options];

// Physics behavior
SCNPhysicsHingeJoint *joint = [SCNPhysicsHingeJoint
  jointWithBodyA:nodeA.physicsBody axisA:[...] anchorA:[...]
  jointWithBodyB:nodeB.physicsBody axisB:[...] anchorB:[...]];
[scene.physicsWorld addBehavior:joint];

// Remove behavior
[scene.physicsWorld removeBehavior:joint];

Constraints

Applied sequentially at render time
Only affect presentation values

aNode.constraints = @[aConstraints, anotherConstraints, ...];

// Custom constraint on a node's transform
aConstrains = [SCNTransformConstrain transformConstrainInWorldSpace:Yes withBlock:
  ^SCNMatrix4(SCNNode *node, SCNMatrix4 transform) {
    transform.m43 = 0.0;
    return transform;
}];

// Makes a node to look at another node: 頭部/眼睛/攝影機都適用
nodeA.constraints = @[SCNLookAtConstraint lookAtConstrainWithTarget:nodeB];

// SCNIKConstraint: 拳頭朝目標物體揮動,連帶會牽動手臂及全身

Scriptability

可與 javascript 整合

  • Javascript bridge
  • Tools
  • Debugging

Rendering

// Color
material.diffuse.contents = [UIColor redColor];

// Image
material.diffuse.contents = @"slate.jpg";

// SKTexture
material.normal.content = [SKTexture textureByGeneratingNormalMap];

// Dynamic content:Video using SKVideo node

// Cube map
/*
        Top
Left   Back   Right   Front
      Bottom
*/
material.reflective.contents = @[@"right.png",@"left.png",...@"front"];

// Custom material: Shader Modifier
// 常用在液體流動材質的呈現或是光暈的表現
material.shaderModifiers = @{<Entry Point>:<GLSL Code>};

// SpriteKit Overlays
sceneView.overlaySKScene = aSKScene;

Effects

Particles

SCNParticleSystem

Shadows

  • static: 效能最好,但不會隨著物體轉動或移動,而跟著變化
  • Dynamic: 最吃效能
  • Projected: 效能介於 static 與 Dynamic 中間,可以指定圖形來呈現陰影
// Static
aMaterial.multiply.contents = aShadowMap;

// Dynamic
aLight.castsShadow = YES;

// Projected
aLight.shadowMode = SCNShadowModeModulated;
aLight.gobo = anImage;

Fog

aScene.fogColor = aColor;
aScene.fogStartDistance = 50;
aScene.fogEndDistance = 100;

Depth of Field

// 景深
aCamera.focalDistance = 16.0;
aCamera.focalBlurRadius = 8.0;

Core Image Filters

aNode.filters = @[gaussianBlurs, distortion, pixelate];

Multi-Pass Effects

合併多種效果

// Load a technique
SCNTechnique *technique = [SCNTechnique techniqueWithDictionary:aDictionary];

// Chain techniques
technique = [SCNTechnique techniqueBySequencingTechniques:@[t1,t2...]];

// Set a technique
aSCNView.technique = technique;

Summary

  • SceneKit available on iOS
  • Casual game ready
  • Full featured rendering
  • Extendable

Morphing

  • Can be loaded from DAE
  • Can be created programmatically
  • topology must match

Skinning

  • Can be loaded from DAE
  • Can not be created programmatically

Performance

Lighting
  • Minimize the number of lights
  • Prefer staic to dynamic shadows
  • Use the "multiply" material property
Texturing
  • Avoid unnecessarily large images
  • Lock ambient with diffuse
    aMaterial.locksAmbientWithDiffuse = YES;
    
  • Use mipmaps
    aMaterial.diffuse.mipFilter = SCNFilterModeLinear;
    
  • Levels of Detail(依遠近呈現不同精細度的物件)

Using Objective-C Classes in Swift

  • Add Objective-C files
  • Create MyApp-Bridging-Header.h
    #import "MyWebViewController.h"
    #import "MyModalWebViewController.h"
    
  • Build Settings -> Objective-C Bridging Header
    /Users/isobar/Desktop/MyApp/MyApp-Bridging-Header.h
    
    swift 這邊不需再 import 可以直接拿來用。

Reference

Install Facebook iOS SDK

#import <FacebookSDK/FacebookSDK.h>

- (void)applicationDidBecomeActive:(UIApplication *)application
{
  // debug
  [FBSettings setLoggingBehavior:[NSSet setWithObjects: FBLoggingBehaviorFBRequests,
                                                        FBLoggingBehaviorFBURLConnections,
                                                        FBLoggingBehaviorAccessTokens,
                                                        FBLoggingBehaviorInformational, nil]];
  [FBSettings setDefaultAppID:APP_ID];
  [FBAppEvents activateApp];
}

App-Info.plist

<key>CFBundleURLTypes</key>
<array>
        <dict>
                <key>CFBundleURLSchemes</key>
                <array>
                        <string>fb778679942XXXXXX</string>
                </array>
        </dict>
</array>
<key>FacebookAppID</key>
<string>778679942XXXXXX</string>
<key>FacebookDisplayName</key>
<string>MY_APP_NAME</string>

Saving NSMutableArray in NSUserDefaults

常犯的錯誤

NSUserDefaults *userDefaults = [NSUserDefaults standardUserDefaults];
NSMutableArray *arr = [userDefaults objectForKey:@"doneForFirstVideo"];

文件有提到,回傳值是 immutable 如下:

Values returned from NSUserDefaults are immutable, even if you set a mutable object as the value.

應該要改成

NSUserDefaults *userDefaults = [NSUserDefaults standardUserDefaults];
NSMutableArray *arr = [[NSMutableArray alloc] initWithArray:[userDefaults objectForKey:@"doneForFirstVideo"]];